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Introduction 

American  national  security  is  based  on  preparedness.  By  ensuring  our  armed  forces’  ability  to  deal  with  any  extant  challenge, 
we  disincentivize  threats  to  our  interests  and  mitigate  the  effects  of  any  attacks  when  perpetrated.  To  truly  be  prepared  for  the 
diverse  body  of  threats  facing  the  U.S.,  from  aggressive  nation-states  to  terrorists  groups,  in  cyber  and  kinetic  domains,  and 
across  land,  sea,  and  air,  weapons  must  be  tested  realistically  in  the  environments  in  which  they  are  to  be  used.  This  is  the 
purpose  of  operational  test  and  evaluation  (OT&E).  It  is  essential  to  assuring  the  men  and  women  we  send  into  combat  can 
win. 

In  my  tenure  as  the  DOD’s  Director  of  Operational  Test  and  Evaluation,  1  have  made  it  my  top  priority  to  ensure  that 
operational  tests  are  adequate,  particularly  regarding  the  realism  of  the  conditions  under  which  the  testing  is  conducted.  In 
doing  this,  I  consider  all  Service-defined  operational  conditions,  including  the  system  operational  envelope,  the  intended 
mission(s),  and  the  range  of  operationally  realistic  kinetic  and  cybersecurity  threats.  Conducting  a  rigorous  and  operationally 
realistic  test  capturing  these  key  parameters  is  the  only  way  to  inform  our  forces  what  weapons  systems  actually  can  and 
cannot  do. 

1  have  also  prioritized  the  objectivity  and  scientific  rigor  of  operational  tests.  By  leveraging  scientific  methodologies 
including  Design  of  Experiments  (DOE),  survey  design,  and  statistical  analyses,  DOT&E  ensures  defensible  and 
efficient  tests  are  conducted  providing  the  critical  information  decision  makers  and  warfighters  require.  Rigorous, 
scientifically-defensible  analyses  of  the  data  ensure  my  reports  tell  the  unvarnished  truth.  This  introduction  summarizes  my 
office’s  continuing  efforts  to  institutionalize  these  methods  in  the  DOD  test  and  evaluation  (T&E)  community. 

Early  stage  testing  can  miss  significant  operationally  relevant  problems  that  are  revealed  during  operational  testing  in  realistic 
environments.  In  FY15,  as  in  previous  years,  OT&E  discovered  problems  missed  during  development  and  in  previous 
testing.  Finding  and  addressing  these  problems  before  production  and  deployment  is  critical,  as  the  only  other  option  is  to 
discover  them  in  combat,  when  the  issues  would  endanger  warfighter  lives.  In  addition,  identifying  and  fixing  these  problems 
once  full-rate  production  is  underway  would  be  a  far  more  expensive  way  to  address  deficiencies,  as  retrofits  are  rarely,  if 
ever,  cheaper  than  fixing  the  problems  before  full-rate  production.  Further  details  on  problem  discovery  during  OT&E  are 
provided  in  a  separate  section  (page  13).  OT&E  also  highlights  and  exposes  previously  known  problems,  as  many  programs 
unfortunately  choose  to  progress  to  operational  testing  with  operationally  significant  unresolved  problems  identified  in  prior 
testing. 

Also  included  in  this  introduction,  I  describe  in  more  detail  several  focus  areas  of  my  office,  including  the  following: 

•  My  continued  emphasis  on  the  need  to  improve  reliability  of  all  weapon  systems  and  my  recent  initiatives  to  include  all 
relevant  information  in  operational  reliability  assessments. 

•  The  recently  released  updated  DOT&E  Test  and  Evaluation  Master  Plan  (TEMP)  Guidebook,  which  provides  new 
guidance  in  my  primary  focus  areas  on  what  substance  and  level  of  detail,  should  be  included  in  TEMPs. 

•  Recent  improvements  made  in  the  area  of  cybersecurity  and  the  need  to  continue  to  emphasize  cybersecurity  as  a  focus 
area  for  all  DOD  systems. 

•  Other  topics  of  interest. 


RIGOROUS,  DEFENSIBLE  TESTING 

In  order  to  provide  rigorous  quantitative  evaluations  of  combat  performance,  and  to  ensure  that  we  fully  utilize  scarce  test 
resources,  1  have  advocated  the  use  of  scientific  test  design  and  statistical  analysis  techniques  for  several  years.  Since 
2009,  there  have  been  substantial  improvements  in  the  use  of  these  techniques  within  the  Services,  specifically  at  each  of 
the  Service  Operational  Test  Agencies  (OTAs).  This  improved  capability  has  provided  the  Department  with  scientifically 
rigorous  test  results  that  identify  what  the  systems  the  Services  are  acquiring  can  and  cannot  do  in  combat.  These  techniques 
have  helped  ensure  adequate  operational  testing;  providing  sufficient  information  to  characterize  combat  performance  across 
the  set  of  operational  scenarios  in  which  the  Services  themselves  state  the  weapon  systems  will  be  used. 
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Both  DOT&E  and  the  Undersecretary  of  Defense  (Acquisition,  Technology  and  Logistics)  (USD(AT&L))  updated 
OSD  policy  and  guidance  to  promote  the  use  of  scientific  approaches  to  test  planning;  in  particular,  the  DOD  Instruction 
5000.02  now  calls  for  universal  employment  of  scientific  approaches  T&E.  Specifically,  the  new  instruction  emphasizes  that 
the  test  program  should  be  designed  to  characterize  combat  mission  capability  across  the  operational  environment  using  an 
appropriately  selected  set  of  factors  and  conditions. 

Warfighters  need  to  know  under  what  conditions  the  system  is  effective  and  when  it  is  not.  This  characterization  is  a  key 
element  of  my  guidance  for  OT&E.  In  OT&E,  characterization  ensures  adequate  information  to  determine  how  combat 
mission  capability  changes  across  the  operational  envelope.  Under  this  concept,  testers  examine  performance  as  a  function 
of  relevant  operational  conditions  and  threat  types.  This  is  in  contrast  to  the  historical  approach  where  test  results  frequently 
have  been  averaged  across  the  operational  envelope.  For  example,  a  metric  such  as  detection  range  was  averaged  across 
all  conditions  and  compared  to  a  single  threshold  requirement  (or  average  historical  performance).  A  simple  average  is  not 
the  best  way  to  evaluate  performance  because  it  fails  to  identify  differences  in  performance  across  the  operational  envelope, 
and  consequently,  it  is  not  informative  to  the  warfighter.  Average  performance  across  all  conditions  masks  variances  in 
performance  across  the  operational  envelope.  An  extreme  example  of  this  I  have  seen  blended  a  100  percent  rating  in  one  set 
of  parameters  with  a  0  percent  rating  in  another,  saying  the  system  was  50  percent  effective  across  conditions.  This  statement 
is  meaningless,  and  the  conditions  under  which  the  system  under  test  is  ineffective  need  to  be  known  by  the  users  and 
developers  of  the  system  so  that  fixes  or  workarounds  can  be  developed. 

1  have  advocated  for  the  use  of  scientific  methods,  including  DOE,  to  ensure  that  this  characterization  is  conducted  as 
efficiently  as  possible.  The  methods  that  I  advocate  not  only  provide  a  rigorous  and  defensible  coverage  of  the  operational 
space,  they  also  allow  us  to  quantify  the  trade-space  between  the  amount  of  testing  and  the  precision  needed  to  answer 
complex  questions  about  system  performance.  They  allow  us  to  know,  before  conducting  the  test,  which  analyses  we  will 
be  able  to  conduct  with  the  data  and  therefore,  what  questions  about  system  performance  we  will  be  able  to  answer.  Finally, 
these  methods  equip  decision  makers  with  the  analytical  tools  to  decide  how  much  testing  is  enough  in  the  context  of 
uncertainty  and  cost  constraints. 

The  Deputy  Assistant  Secretary  of  Defense  Developmental  Test  and  Evaluation  (DASD(DT&E))  has  advocated  the  use 
of  these  methods  through  his  Scientific  Test  and  Analysis  Techniques  (STAT)  T&E  Center  of  Excellence  (COE),  which 
employs  qualified  statistics  experts  to  aid  acquisition  program  managers  in  applying  advanced  statistical  techniques  in 
developmental  testing.  The  STAT  T&E  COE  helps  program  managers  plan  and  execute  more  efficient  and  effective  tests 
beginning  with  early  developmental  testing.  Initially  20  Acquisition  Category  I  programs  were  partnered  with  the  COE.  To 
date,  36  programs  have  had  dedicated  COE  support  for  development  of  test  strategies,  mentoring,  or  training.  The  COE  is 
envisioned  to  eventually  be  funded  by  the  Services’  in  order  to  expand  in  size  and  also  provide  support  to  program  managers 
in  smaller  acquisition  programs.  I  encourage  all  program  offices  to  ensure  that  they  have  access  to  such  a  knowledge  source. 

As  a  community,  we  should  always  strive  to  improve  our  test  methods.  While  1  have  seen  improvements  in  several  areas, 
continued  improvement  is  possible.  Important  future  focus  areas  include:  statistical  analytic  techniques  to  examine  test 
results,  improving  surveys  in  testing,  validation  of  models  and  simulations,  and  using  all  the  appropriate  information  to 
maximize  the  information  available  to  decision  makers  and  operators. 

Statistical  Analytic  Techniques 

It  is  not  sufficient  to  employ  statistical  methods  only  in  the  test  design  process;  corresponding  analysis  methods  should 
be  employed  in  the  evaluation  of  system  performance,  otherwise  we  risk  missing  important  conclusions.  Using  statistical 
analysis  methods  instead  of  conventional  approaches  to  data  analysis,  we  have  been  able  to  learn  more  from  tests  without 
necessarily  increasing  their  size  and  cost.  In  all  of  my  reports,  my  staff  uses  rigorous  statistical  analysis  methods  to  provide 
more  information  from  operational  tests  than  ever  before.  In  the  past  few  years,  my  staff  has  used  these  analysis  techniques 
to  identify  areas  of  performance  shortfalls.  For  example,  in  the  operational  test  and  of  the  Multi-Spectral  Targeting  System, 
which  is  intended  to  enable  helicopters  to  target  small-fast  boats  and  employ  F1ELLFIRE  missiles,  a  logistic  regression 
of  the  test  results  revealed  a  significant  interaction  between  two  factors  that  resulted  in  performance  falling  well  below 
the  required  value  in  one  of  the  scenarios,  suggesting  the  need  for  a  potential  system  algorithm  improvement.  In  another 
example,  the  operational  testing  of  the  AN/TPQ-53  Counterfire  radar,  showed  how  performance  degraded  as  a  function  of 
range  and  projectile  elevation.  This  analysis  was  especially  useful  because  in  this  case  testers  did  not  control  all  factors 
likely  to  affect  performance  in  order  to  maintain  operational  realism.  Regression  techniques  enabled  DOT&E  to  determine 
causes  of  performance  degradations  across  multiple  operating  modes,  even  with  highly  unbalanced  data.  Finally,  we  are 
using  statistical  analysis  techniques  to  show  statistically  significant  improvements  between  incrementally  improved  versions 
of  systems.  In  the  operational  testing  of  the  Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion  (A-RCI)  sonar  system  an 
in-lab  portion  of  testing  was  added  to  the  traditional  at-sea  testing  to  evaluate  operator  detection  capabilities  across  a  range  of 
environments  and  targets.  Statistical  analysis  techniques  (coupled  with  a  robust  experimental  design)  showed  a  statistically 
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significant  improvement  in  the  software  build  over  the  legacy  build  and  allowed  us  to  definitively  claim  that  the  improvement 
was  universal  across  all  operating  conditions.  It  is  important  to  note  that  if  DOT&E  had  not  pushed  for  these  more  rigorous 
analyses,  all  of  these  results  would  have  been  missed. 

Rigorous  methods  should  also  be  used  for  suitability  analyses.  In  the  past  year,  1  have  put  a  larger  emphasis  on  the  rigorous 
analysis  of  survey  and  reliability  data.  One  notable  example  of  this  is  the  reliability  assessment  conducted  for  the  Littoral 
Combat  Ship  (LCS).  The  LCS  reliability  requirement  as  stated  would  have  been  nearly  impossible  to  test,  requiring  a  core 
mission  reliability  of  0.80  for  a  720-hour  mission.  Instead,  my  office  focused  on  critical  sub-systems  that  contributed  to  the 
core  mission.  Using  Bayesian  methodologies  and  series  system  models  we  were  able  to  assess  the  core  mission  reliability 
defensibly,  providing  reasonable  interval  estimates  of  the  reliability  even  in  cases  where  the  critical  sub-systems  had  different 
usage  rates  and  zero  failures.  This  type  of  analysis  also  lays  the  groundwork  for  how  different  sources  of  information 
discussed  below  can  be  used  to  evaluate  system  reliability  and  performance. 

Unfortunately,  the  implementation  of  rigorous  statistical  techniques  is  still  far  from  widespread  across  all  DOD  T&E 
communities.  Overall,  statistical  analysis  methods  such  as  logistic  regression  and  analysis  of  variance,  which  supported  the 
above  discoveries,  are  underused.  Until  they  are  routinely  employed  in  the  analysis  of  T&E  data,  the  OT&E  community  will 
miss  opportunities  to  identify  important  performance  results  and  truly  understand  system  capability.  Furthermore,  we  are  not 
currently  leveraging  these  methods  in  a  sequential  fashion  to  improve  knowledge  as  we  move  from  developmental  testing 
to  operational  testing.  Knowledge  about  the  most  important  factors  from  developmental  testing  will  improve  our  ability  to 
clearly  define  an  adequate  operational  test  that  avoids  the  unnecessary  expenditure  of  resources. 

Survey  Design  and  Analysis 

In  2015, 1  issued  additional  guidance  on  the  design  and  use  of  surveys  in  OT&E.  Surveys  provide  valuable  quantitative  and 
qualitative  information  about  the  opinions  of  operators  and  maintainers  as  they  employ  and  maintain  weapon  systems  in  an 
operationally  realistic  test  environment.  An  objective  measurement  of  these  opinions  is  an  essential  element  of  my  evaluation 
of  operational  effectiveness  and  suitability.  However,  I  have  noted  that  many  of  the  surveys  used  in  OT&E  are  of  such  poor 
quality  they  can  actually  hinder  my  ability  to  objectively  evaluate  the  system.  My  office  has  worked  closely  with  the  Service 
OTAs  to  improve  the  quality  of  surveys  used  in  operational  testing. 

Custom  surveys,  established  surveys  (e.g.,  NASA  workload  questionnaire),  interviews,  and  focus  groups  all  have  important 
roles  in  OT&E.  For  example,  focus  groups  are  often  essential  venues  to  elicit  operator  opinions;  however,  focus  groups 
should  not  be  the  sole  source  of  operator  opinion  data.  Focus  groups  can  be  affected  by  group  dynamics  and  therefore  should 
be  used  to  obtain  diagnostic  information  rather  than  quantitative  information.  To  maximize  the  usefulness  of  focus  groups, 
the  test  team  should  examine  the  survey  responses  immediately  after  administering  them  to  look  for  trends.  These  initial 
results  can  then  be  used  to  help  guide  the  focus  group  questioning  which  should  occur  after  the  written  surveys  but  as  soon  as 
possible  to  ensure  impressions  are  still  fresh  in  the  user’s  minds. 

All  of  the  OTAs  are  currently  working  on  improving  their  own  guidance  on  the  use  of  surveys  in  OT&E.  Once  the  scientific 
best  practices  I  have  advocated  for  are  incorporated,  I  expect  future  evaluations  to  include  better  quality  and  usable  survey 
results. 

Validation  of  Modeling  and  Simulations 

Modeling  and  simulation  (M&S)  can  and  often  does  provide  complementary  information  that  is  useful  in  my  evaluations 
of  operational  effectiveness,  suitability,  and  survivability.  For  example,  there  are  cases  in  which  not  all  of  the  important 
aspects  of  weapon  system  effectiveness  or  system  survivability  can  be  evaluated  in  an  operationally  realistic  environment 
due  to  safety,  cost,  or  other  constraints.  In  these  cases,  M&S  provides  valuable  information  to  my  assessment.  However, 
for  M&S  to  be  useful  it  must  be  rigorously  validated  to  ensure  that  the  simulations  adequately  represent  the  real-world 
performance  under  the  conditions  of  its  intended  use  (at  a  specific  level  of  accuracy).  A  model  that  is  validated  under  one  set 
of  operational  conditions  may  not  be  valid  under  other  sets  of  operational  conditions. 

Since  my  assessment  of  operational  effectiveness  includes  the  characterization  of  combat  mission  capability  across  the 
operational  envelope,  validation  methods  must  ensure  that  M&S  is  valid  across  that  operational  envelope.  We  need  to 
explore  new  scientific  methods  for  validation  that  allow  me  to  characterize  where  the  M&S  provides  useful  information  to 
my  assessments  and  where  models  do  not  represent  the  real-world  conditions  to  a  high  enough  level  of  accuracy.  Historical 
methods  of  rolling  up  accuracy  of  the  M&S  across  a  variety  of  conditions  do  not  provide  this  level  of  fidelity  and  must  be 
improved  upon  using  state-of-the-art  scientific  methods. 

In  my  recent  review  of  TEMPs  that  propose  M&S  as  a  key  aspect  of  operational  testing,  I  reviewed  the  selection  of  M&S 
points  and  the  validation  methods  with  the  same  scrutiny  as  the  proposed  live  operational  test  points  in  order  to  ensure 
adequacy. 
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Using  All  Information  in  Operational  Evaluations 

Operational  testing  occurs  under  realistic  combat  conditions,  including  operational  scenarios  typical  of  a  system’s 
employment  in  combat,  realistic  threat  forces,  and  employment  of  the  systems  under  test  by  typical  users  rather  than  by 
hand-picked  or  contractor  crews.  History  has  shown  us  that  emphasizing  operational  realism  is  essential  in  identifying 
critical  system  performance  problems,  many  of  which  are  only  discoverable  in  an  operationally  realistic  environment. 
However,  operational  testing  is  limited  in  that  it  typically  spans  a  short  period  of  time  compared  to  the  rest  of  the  testing 
continuum.  In  many  cases,  it  is  beneficial  to  consider  other  test  data  in  an  operational  evaluation.  In  doing  so,  we  must 
account  for  the  fact  that  these  additional  data  were  collected  under  less  operationally  realistic  conditions. 

In  cases  where  other  test  data,  especially  that  from  operationally  realistic  developmental  testing,  operational  assessments, 
and  M&S,  provide  additional  information  we  should  use  state-of-the-art  analysis  methods  to  include  that  information  in  our 
analyses.  However,  it  is  also  essential  that  we  avoid  biasing  the  operationally  realistic  results  in  such  analyses.  Thoughtful 
application  of  statistical  models,  especially  Bayesian  models,  has  proven  useful  in  this  regard. 


IMPROVING  SYSTEM  RELIABILITY 


Many  defense  systems  continue  to  demonstrate  poor  reliability  in  operational  testing.  As  shown  in  Figure  1,  only  9  of 
24  (38  percent)  systems  that  had  an  Initial  Operational  Test  and  Evaluation  (IOT&E)  or  Follow-on  Operational  Test  and 
Evaluation  (FOT&E)  in  FY15  met  their  reliability  requirements.  The  remaining  15  systems  either  failed  to  meet  their 
requirements  (29  percent),  met  their  requirements  on  some  (but  not  all)  platforms  on  which  they  were  integrated  (8  percent), 
or  could  not  be  assessed  because  of  limited  test  data  or  the  absence  of  a  reliability  requirement.  In  four  instances  where  the 
system  failed  to  meet  its  reliability  requirement  or  did  not  have  a  reliability  requirement,  DOT&E  assessed  that  the  reliability 
demonstrated  in  testing  was  sufficient  to  support  operational  missions  resulting  in  13  of  24  (54  percent)  programs  being 
assessed  as  operationally  reliable. 


- . 54% 


B  Assessment  of  Whether  System  Met 
Reliability  Requirement(s) 

□  DOT&E  Assessment  of  Mission  Reliability 


Various  policies  have  been  established  to 
improve  reliability  performance.  Most 
recently,  the  January  2015  update  to  the 
DOD  5000.02  codified  the  need  for  programs 
to  employ  best  practices  in  reliability  growth 
planning.  The  instruction  requires  program 
managers  to  formulate  a  comprehensive 
reliability  and  maintainability  program  that 
is  part  of  the  systems  engineering  process, 
assess  the  reliability  growth  required  for  the 
system  to  achieve  its  reliability  threshold 
during  IOT&E,  and  report  the  results  of 
that  assessment  to  the  Milestone  Decision 
Authority  at  Milestone  C. 

Since  my  office  began  monitoring  reliability  in  2005, 
programs  have  increasingly  complied  with  these 
policies,  but  this  has  not  yet  translated  to  improved  reliability  performance.  Common  reasons  why  programs  fail  reliability 
requirements  include  lack  of  a  design  for  reliability  effort  during  the  design  phase;  unrealistic  requirements  that  are  too  large 
relative  to  comparable  systems;  lack  of  contractual  and  systems  engineering  support;  insufficient  developmental  test  time 
to  identify  and  correct  failure  modes;  absence  of,  or  disagreement  on,  reliability  scoring  procedures;  or  failure  to  correct 
significant  reliability  problems  discovered  in  developmental  testing  prior  to  operational  testing. 


Met 


Did  not  Meet 


Requirement(s)  Requirement(s) 


Met  on  Some 
(but  not  all) 
Platforms 


Unknown  or  Not 
Applicable 


FIGURE  1 .  RELIABILITY  ASSESSMENT  FOR  24  SYSTEMS  THAT  HAD  AN 
IOT&E  OR  FOT&E  IN  FY15 


Despite  these  shortfalls,  there  is  some  evidence  that  programs  with  a  reliability  Key  Performance  Parameter  (KPP)  are  more 
likely  to  meet  their  reliability  requirements.  A  20 14  National  Academy  of  Sciences  report  commissioned  by  myself  and 
Mr.  Frank  Kendall  (USD(AT&L)  recommended  programs  develop  a  reliability  KPP  and  ensure  that  all  proposals  explicitly 
designate  funds  for  reliability  improvement  activities.  1  To  follow-up  on  this  recommendation,  my  office  reviewed  the 
requirements  documents  for  programs  that  conducted  an  operational  test  in  2014.  Of  the  34  programs  that  had  an  IOT&E  or 
FOT&E  in  FY14  and  had  a  reliability  requirement  in  their  Capability  Development  Document  (CDD),  8  had  a  reliability  KPP 
and  26  did  not.  Seven  of  the  eight  programs  (88  percent)  with  reliability  KPPs  achieved  their  reliability  requirements  while 


1 .  National  Academy  of  Sciences,  Reliability  Growth:  Enhancing  Defense  System  Reliability,  2014. 
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only  11  of  the  26  (42  percent)  programs  without  reliability  KPPs  achieved  their  requirement.  This  initial  result  provides 
limited  evidence  that  requiring  reliability  KPPs  may  be  a  good  policy  change  for  ensuring  programs  take  the  need  for  reliable 
systems  seriously. 

In  the  same  annual  review  on  reliability,  my  office  noted  that  over  a  quarter  (27  percent)  of  programs  had  operational  test 
lengths  that  were  shorter  in  duration  than  their  reliability  requirement.  As  part  of  my  ongoing  effort  to  ensure  that  testing 
is  done  as  efficiently  as  possible,  I  have  continually  encouraged  programs  to  intelligently  use  information  from  all  phases 
of  test,  particularly  when  assessing  reliability.  Similar  to  the  assessment  of  other  system  capabilities,  it  is  important  to 
understand  the  risks  to  both  the  government  and  the  contractor  when  determining  the  appropriate  length  of  a  test.  Overly 
simple  rules  of  thumb  such  as  testing  for  duration  equal  to  three  times  the  reliability  requirement  often  lead  to  inconclusive 
assessments.  In  other  cases,  system  reliability  requirements  can  be  so  high  that  a  test  adequate  for  assessing  effectiveness 
would  only  permit  a  limited  assessment  of  reliability.  This  situation,  in  particular,  benefits  from  the  intelligent  incorporation 
of  developmental  and  early  operational  test  data  in  the  final  reliability  assessments.  It  is  crucial  to  note  that  this  does  not 
mean  simply  adding  developmental  test  data  to  operational  test  data.  A  rigorous  statistical  approach  that  accounts  for  the 
differences  in  test  environments  is  necessary. 

When  a  program  intends  to  use  developmental  test  data  to  support  an  operational  assessment,  it  is  crucial  to  involve  the 
operational  test  community  early  in  the  data  scoring  process.  Scoring  conferences,  used  extensively  by  both  the  Air  Force 
and  the  Army,  provide  a  forum  for  stakeholders  to  discuss  reliability,  and  I  recommend  that  all  programs  use  them.  Even  if 
a  program  does  not  intend  to  use  developmental  test  data  to  supplement  the  operational  assessment,  including  operational 
testers  in  scoring  conferences  for  developmental  tests  provides  the  Program  Office  a  better  understanding  of  how  issues 
observed  in  developmental  testing  are  likely  to  effect  the  system’s  reliability  assessment  in  subsequent  operational  testing. 
This  helps  program  offices  identify  priority  corrective  actions.  1  have  updated  my  guidance  on  reliability  test  planning  in 
the  recently  updated  DOT&E  TEMP  Guidebook  to  address  my  desire  to  incorporate  all  relevant  information  into  operational 
reliability  assessments. 


TEMP  GUIDEBOOK  3.0 

Throughout  my  tenure,  I  have  always  strived  to  provide  clear  guidance  on  my  expectations.  This  year  my  office  updated 
the  DOT&E  TEMP  Guidebook  to  complement  the  January  2015  version  of  DOD  Instruction  5000.02.  While  the  updates 
also  included  formatting  updates,  strict  or  immediate  adherence  to  the  new  TEMP  format  is  not  required  as  my  evaluation 
of  TEMP  adequacy  is  based  on  the  TEMP's  content,  not  the  format.  The  TEMP  Guidebook  3.0  follows  the  updated  DOD 
5000.02  TEMP  organization;  there  are  bold  blue  font  callouts  with  links  to  DOT&E  guidance  and  examples.  The  callouts 
have  been  placed  throughout  TEMP  Guidebook  3.0  at  locations  where  DOT&E  and  other  applicable  policies  apply.  The 
combination  of  guidance  and  examples  is  intended  to  highlight  areas  of  emphasis  to  me,  and  provide  clear  examples  how  my 
guidance  should  be  interpreted. 

There  are  several  key  content  areas  that  my  office  revised  in  this  third  iteration  of  the  TEMP  Guidebook  based  on  lessons 
learned  over  the  past  several  years.  The  primary  areas  where  substantive  updates  were  made  were  the  creation  of  an 
operational  evaluation  framework,  methods  for  combining  information  from  multiple  phases  of  testing,  reliability  test 
planning,  and  cybersecurity. 

I  have  also  expanded  my  guidance  on  the  use  of  developmental  test  data  for  operational  test  evaluation.  In  the  current  fiscal 
climate,  it  is  important  we  test  enough  to  provide  the  warfighter  with  valuable  information  on  system  capability  without 
testing  too  much.  1  have  taken  every  opportunity  to  use  all  information  available  to  me  to  ensure  we  provide  valuable 
information  as  efficiently  as  possible.  The  Integrated  Testing  section  and  the  Bayesian  guidance  section  capture  best  practices 
for  leveraging  all  available  information  while  still  ensuring  operational  assessments  reflect  performance  in  the  operational 
environment. 

There  is  a  new  section  on  reliability  test  planning,  which  is  distinctly  different  from  the  reliability  growth  section.  This  new 
section  provides  clear  guidance  on  my  expectations  for  planning  reliability  tests  as  well  as  what  information  I  expect  to  be  in 
a  reliability  growth  program. 

Additionally,  TEMP  Guidebook  3.0  contains  expanded  guidance  and  examples  for  implementation  of  the  DOT&E 
memorandum,  “Procedures  for  Operational  Test  and  Evaluation  of  Cybersecurity  in  Acquisition  Programs”  dated 
August  1,  2014.  These  examples  are  based  on  lessons  learned  from  cybersecurity  test  successes  and  challenges  in  the  past 
year  of  implementing  the  2014  DOT&E  cybersecurity  procedures  memorandum. 
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CYBERSECURITY 

DOT&E  observed  improvements  in  several  cybersecurity  areas  within  the  DOD  this  past  year;  however,  operational 
missions  and  systems  remain  vulnerable  to  cyber-attack.  Observed  improvements  during  training  exercises  include 
enhanced  protection  of  network  elements,  greater  challenges  for  cyber  opposing  forces  attempting  to  access  networks,  and 
growing  awareness  by  DOD  leadership  that  cyber-attacks  can  degrade  key  systems  and  critical  missions.  In  some  networks, 
vulnerabilities  routinely  available  elsewhere  were  mitigated  by  timely  upgrades  and  software  patches.  Operational  tests  of 
isolated  systems  experienced  much  less  success  in  preventing  and  detecting  cyber  intrusions  highlighting  the  importance  of 
cyber  defense-in-depth.  A  layered  approach  to  stop  primary  attack  vectors,  such  as  phishing,  proved  effective  at  defending 
some  networks.  Application  whitelisting,  where  network  defenders  allow  only  “known  good”  applications  to  operate  on  a 
network,  also  hindered  the  cyber  opposing  force  from  expanding  its  foothold  in  the  network.  However,  these  improvements 
were  insufficient  to  ensure  that  networks  and  systems  can  continue  to  support  DOD  missions  in  the  presence  of  a  cyber 
adversary. 

In  FY15  operational  tests  and  exercise  assessments,  cyber  opposing  forces  frequently  attained  a  position  to  deliver  cyber 
effects  that  could  degrade  operational  missions,  often  significantly.  Unfortunately,  exercise  and  test  control  authorities 
seldom  permitted  aggressive  cyber-attacks  to  affect  systems  and  networks,  or  allowed  non-cyber  forces  to  exploit 
compromised  information  in  their  operations.  These  restrictions  limit  insights  on  both  the  scope  and  duration  of  associated 
mission  effects  and  preclude  the  opportunity  for  training  in  representative  cyber-contested  conditions.  Acquisition  programs, 
Combatant  Commands,  Services,  and  cyber  defenders  need  realistic  operational  tests  and  training  events  that  include 
cyber-attacks  and  mission  effects  representative  of  those  expected  from  advanced  capability  cyber  adversaries. 

The  demand  on  DOD-certified  Red  Teams,  which  are  the  core  of  the  cyber  opposing  forces  teams,  has  more  than  doubled 
in  the  past  three  years.  In  the  same  timeframe,  the  Cyber  Mission  Force  and  private  sector  have  hired  away  members  of 
Red  Teams,  resulting  in  staffing  shortfalls  during  a  time  with  increasing  demand.  To  reduce  administrative  overhead  and 
increase  the  realism  in  portraying  cyber  threats,  DOT&E  worked  with  U.S.  Pacific  Command,  U.S.  Northern  Command, 

U.S.  Strategic  Command,  and  U.S.  Cyber  Command  to  establish  permissions  for  continuous  Red  Team  operations  on  selected 
DOD  networks  and  systems.  DOT&E  also  helped  Red  Teams  access  advanced  cyber  capabilities  so  that  they  can  better 
emulate  advanced  capability  cyber  threats.  However,  these  efforts  alone  will  not  offset  the  Red  Team  staffing  and  capability 
shortfalls,  which  the  DOD  must  address  to  retain  the  ability  to  assess  DOD  systems  and  train  Service  members  against 
realistic  cyber  threats. 


ADDITIONAL  TOPICS  OF  INTEREST 

In  this  section,  I  provide  details  on  specific  test  resources  and  test  venues  that  have  had  significant  action  on  my  part  this  year. 
For  more  details  on  the  Multi-Stage  Supersonic  Target  (MSST),  self-defense  test  ship  (SDTS),  Radar  Signal  Emitters  (RSE), 
Warrior  Injury  Assessment  Manikin  (WIAMan),  and  Fifth-Generation  Aerial  Target  (5GAT),  see  the  Resources  section  of  this 
Annual  Report  (page  397). 


DOT&E  Staffing 

The  FY08  National  Defense  Authorization  Act  (NDAA) 
expressed  concern  about  the  adequacy  of  DOT&E  staffing 
and  directed  a  manpower  study  be  conducted.  As  a  result  of 
that  study,  the  Secretary  of  Defense  authorized  22  additional 
government  billets  for  DOT&E,  increasing  civilian 
authorizations  from  54  to  76.  Subsequently,  in  FY10,  the 
DOD  evaluated  contractor  support  Department-wide  and 
authorized  in-sourcing  of  inherently  government  functions 
while  directing  a  reduction  in  the  levels  of  contractor 
support  for  headquarters  organizations.  As  a  result,  DOT&E 
in-sourced  1 7  inherently  government  positions  and  reduced 
contractor  support  by  a  total  of  47  (from  72  in  2008  to  25  in 
2015  and  beyond).  Multiple  OSD  efficiency  reviews  further 
reduced  DOT&E  civilian  authorizations  from  93  to  67  by 
FY20. 


FIGURE  2.  DOT&E  CIVILIAN  AND  CONTRACTOR  STAFF  PROJECTION 
BETWEEN  2010 -2020 


Between  2010  and  2020,  DOT&E  civilian  and  contractor  personnel  will  shrink  by  42  percent,  and  DOT&E  anticipates  further 
reductions  in  budgets  and/or  manpower  authorizations.  It  is  noteworthy  that  DOT&E,  unlike  other  headquarters  staffs,  did 
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not  receive  any  additional  manpower  or  funding  to  support  the  missions  of  Operation  Iraqi  Freedom  (OIF)  and  Operation 
Enduring  Freedom  (OEF).  Because  headquarters  staff  reductions  Department-wide  are  intended  to  reduce  those  staffs 
that  grew  larger  to  support  OEF  and  OIF,  the  impact  to  DOT&E  staffing  is  especially  significant.  To  preserve  its  Title  10 
responsibilities,  it  is  likely  that  DOT&E  will  have  to  terminate  some  non-core,  non-Title  10  activities. 

Multi-Stage  Supersonic  Target  (MSST) 

The  Navy’s  MSST  program  was  intended  to  provide  a  threat  representative  surrogate  for  a  specific  class  of  Anti-Ship  Cruise 
Missiles  (ASCMs).  Unfortunately,  the  MSST  program,  originally  intended  to  cost  $297  Million,  ballooned  to  $962  Million 
and  was  nearly  five  years  behind  schedule.  Moreover,  recent  analysis  by  the  Navy’s  intelligence  community  indicated  the 
target,  if  completed,  would  likely  have  been  a  poor  surrogate  for  the  threats  it  was  intended  to  emulate.  For  these  reasons,  the 
Navy  directed  that  the  program  be  terminated. 

I  agree  with  the  Navy’s  decision  to  terminate  the  MSST  program.  I  also  strongly  recommended  to  the  Navy  that  it  not  pursue 
a  segmented,  highly  artificial  test  approach  as  a  substitute  for  the  MSST  that  the  Navy  estimated  would  have  cost  more  than 
$700  Million  to  implement.  The  artificialities  of  the  alternative  proposed  by  the  Navy  would  have  hopelessly  confounded  the 
interpretation  of  any  results  obtained  from  its  use,  making  it  unwise,  unwarranted,  and  a  waste  of  resources.  Nevertheless, 
without  a  threat  representative  surrogate  for  the  threats  the  MSST  was  intended  to  emulate,  I  will  not  be  able  to  assess  the 
ability  of  Navy  surface  combatants  to  defend  against  such  threats. 

Aegis  Self-Defense  Test  Ship  (SDTS) 

The  Navy’s  Aegis  cruisers  and  destroyers  are  charged  with  defending  our  Carrier  Strike  and  Amphibious  Ready  Groups 
against  ASCM  attacks.  Without  such  a  defense,  the  self-defense  systems  on  our  carriers  and  amphibious  ships  may  be 
overwhelmed.  It  is  thus  critical  that  our  Aegis  ships  be  able  to  defend  themselves  against  ASCM  attacks  so  they  can  survive 
and  complete  their  air-defense  missions.  These  facts  are  reflected  in  the  self-defense  requirements  for  all  new  ship  classes 
and  combat  system  elements  to  include  the  Navy’s  new  flight  of  DDG  5 1  destroyers  (DDG  5 1  Flight  III),  the  Air  and  Missile 
Defense  Radar  (AMDR)  that  is  to  be  installed  on  DDG  5 1  Flight  III,  the  upgraded  Aegis  Weapon  System  planned  for  DDG  5 1 
Flight  III,  and  the  Block  2  upgrade  to  the  Evolved  SeaSparrow  Missile  (ESSM  Block  2). 

Operationally  realistic  testing  of  DDG  5 1  Flight  III,  AMDR,  the  Aegis  Weapons  System,  and  ESSM  Block  2  requires 
demonstrating  the  ship’s  combat  system’s  ability  to  defeat  raids  of  ASCMs  including  a  particularly  menacing  and  proliferating 
set  of  threats— supersonic  ASCMs  flying  directly  at  the  ship  (stream  raids).  Navy  sea-range  safety  restrictions  do  not  permit 
ASCM  surrogates  to  be  flown  directly  at  crewed  ships;  even  with  a  cross-range  aim-point,  the  surrogate  threats  cannot  fly 
within  the  ranges  necessary  to  test  the  ship’s  self-defense  combat  system.  Amphibious  ship  classes  and  aircraft  carriers  have 
used  a  crewless  SDTS  in  combination  with  live  firings  and  M&S  to  evaluate  their  self-defense  systems.  However,  the  Aegis 
combat  system  has  never  been  installed  on  a  test  ship.  For  nearly  three  years,  my  office  has  engaged  the  Navy  regarding  the 
need  for  an  AMDR-  and  Aegis-equipped  SDTS.  In  doing  so,  my  office  has  detailed  numerous  problems  found  on  other  Navy 
surface  combatants  only  as  a  direct  result  of  testing  on  a  SDTS.  Without  those  tests,  critical  failure  modes  would  not  have 
been  found  and  could  not  have  been  corrected. 

In  2015,  OSD  Cost  Analysis  Performance  Assessment  (CAPE)  studied  various  options  for  acquiring  an  Aegis-  and 
AMDR-equipped  SDTS.  The  CAPE  study,  which  was  based  on  Navy  cost  data,  showed  that  an  appropriately-equipped 
SDTS  could  be  acquired  for  $320  Million.  DOT&E  has  raised  this  issue  to  the  Secretary  and  Deputy  Secretary  for  resolution 
in  the  FY17  program  and  budget  review.  Meanwhile,  DOT&E  continues  to  work  with  the  Navy  to  develop  an  integrated  test 
plan  for  live  firings  using  crewed  ships,  the  SDTS  (if  available),  and  M&S. 

Radar  Signal  Emitters  (RSE) 

In  order  to  improve  realism  of  electronic  warfare  threats  at  open  air  ranges,  DOT&E  is  collaborating  with  the  Test  Resource 
Management  Center  (TRMC)  and  Army  Threat  Systems  Management  Office  (TSMO),  to  procure  a  fleet  of  mobile, 
programmable  radar  signal  emulators  (RSEs)  designed  to  replicate  a  wide  variety  of  modern,  ground-based  threat  air  defense 
radars.  These  test  assets  are  essential  for  creating  operationally  realistic,  multi-layered  air  defense  scenarios  for  open-air 
testing  of  many  new  systems  that  are  required  to  operate  in  an  Anti-Access  Air  Denial  (A2AD)  environment.  These  systems 
include  the  Joint  Strike  Fighter  (JSF),  F-22,  B-2,  Long-Range  Strike  Bomber,  and  the  Next  Generation  Jammer  for  the 
EA-18G,  as  well  as  others.  The  first  two  RSEs  are  schedule  to  be  delivered  to  the  Nevada  Test  and  Training  Range  (NTTR) 
for  testing  and  integration  in  FY16.  A  total  of  16  systems  are  under  contract  and  scheduled  to  be  delivered  and  integrated  at 
Air  Force  and  Navy  open-air  test  ranges. 

Now  that  the  JSF  Program  Office  has  decided  to  discontinue  the  Lock  heed  Martin  Verification  Simulation,  a  high-fidelity 
manned  simulation  that  had  been  central  to  JSF’s  operational  test  plans,  the  ability  of  open-air  testing  to  replicate  more 
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realistic  and  stressing  operational  environments  is  paramount.  Having  the  RSEs  integrated  on  the  test  ranges  and  available 
for  the  JSF  IOT&E  is  essential. 

Significant  progress  was  made  this  year  on  the  development,  production,  and  planning  for  testing  and  range  integration  of 
the  first  two  RSEs.  Each  RSE  is  capable  of  high-fidelity  emulation  of  the  output  power,  signal  parameters,  and  performance 
of  long-range  surface-to-air  missile  radars,  and  is  mounted  on  its  own  highway-certified  and  range-road-capable  trailer  with 
integral  cooling  for  all  weather  operability.  Once  delivered  to  NTTR,  these  systems  will  each  be  paired  with  a  tow  vehicle 
that  incorporates  a  generator  for  powering  the  RSE,  communications  equipment  for  connecting  to  range  networks,  and  an 
operator  control  cabin.  The  RSEs  are  rapidly  reprogrammable  and  capable  of  emulating  the  signals  of  a  wide  variety  of 
radars  found  in  modern  air  defense  environments.  They  employ  active  electronically-steered  array  radar  technology  with 
high-powered,  high-efficiency  transmit  and  receive  modules. 

With  close  cooperation  of  the  Air  Force  NTTR  range  personnel,  the  integration  and  implementation  of  the  RSEs  for  the  JSF 
IOT&E  was  defined.  Several  test  events  are  currently  being  planned  for  initial  check  out.  Operational  testing  of  the  RSEs  is 
expected  to  begin  by  the  end  of  2016. 

Additionally,  we  are  now  working  closely  with  the  Navy  range  personnel  (Point  Mugu  Sea  Test  Range)  to  implement 
enhancements  at  that  range  necessary  to  incorporate  the  RSEs.  The  Navy  will  eventually  take  ownership  of  5  RSEs  and  the 
Air  Force  the  other  11  for  the  purposes  of  operations  and  maintenance.  However,  the  mobility  of  the  systems  is  such  that  any 
or  all  of  the  RSEs  would  be  available  for  any  test  program  that  requires  them,  and  they  are  readily  transportable  by  air  (C-17 
or  C-130)  or  over  the  road  to  a  variety  of  test  ranges. 

Warrior  Injury  Assessment  Manikin  (WlAMan) 

There  have  been  over  23,000  casualties  from  underbody  blast  (UBB)  events  due  to  improvised  explosive  devices  (IEDs)  in 
the  Iraq  and  Afghanistan  conflicts;  furthermore,  the  UBB  threat  has  been  an  effective  enemy  tactic  over  the  past  decade  and  a 
half,  and  it  is  likely  to  remain  so.  The  need  to  protect  our  Service  members  from  this  threat  in  the  future  is  clearly  reflected  in 
the  force  protection  requirements  developed  by  the  Services  for  their  ongoing  combat  and  tactical  wheeled  vehicle  programs. 
The  Army  has  spent  over  $2  Billion  to  retrofit  existing  vehicles  with  UBB  protection.  New  vehicles  such  as  the  Joint  Light 
Tactical  Vehicle,  the  Amphibious  Combat  Vehicle,  and  the  Mobile  Protected  Firepower  Light  Tank  are  being  procured  with 
requirements  to  protect  occupants  against  UBB  threats.  However,  the  Department  remains  without  an  adequate  test  device 
and  scientifically-defensible  injury  criteria  to  effectively  evaluate  the  protection  provided  by  our  combat  and  tactical  wheeled 
vehicles. 

The  Department’s  inability  to  assess  injuries  due  to  UBB  events  was  made  clear  during  the  early  (2007  -  2009)  LFT&E 
of  the  Mine-Resistant  Ambush  Protected  (MRAP)  vehicles,  when  the  Army  could  not  evaluate  differences  in  the  degree  of 
force  protection  provided  to  occupants  by  the  different  MRAP  variants  due  to  non-biofidelic  instrumentation  and  poor  injury 
assessment  capability.  The  DOT&E  MRAP  assessment,  published  in  2010,  highlighted  these  test  resource  deficiencies. 
Despite  these  shortcomings,  the  same  ineffective  instrumentation  and  injury  criteria  used  in  those  tests  remain  in  use  today. 

As  part  of  a  retrospective  review  of  MRAP  procurement  and  performance,  the  DOD  directed  a  status  review  of  UBB  M&S 
to  determine  if  an  enhanced  UBB  M&S  capability  could  have  identified  the  MRAP  performance  differences  prior  to  the 
publication  of  the  DOT&E  report.  The  review  identified  10  major  gaps  in  the  Department’s  capability  to  accurately  model 
the  effects  of  UBB;  the  top  three  gaps  were  all  associated  with  the  shortcomings  in  test  instrumentation  and  criteria  to  assess 
human  injury  in  the  UBB  environment.  This  study  highlighted  that  the  current  T&E  techniques  used  to  address  occupant 
injuries  in  UBB  LFT&E  (using  automotive  crash  test  dummies  and  injury  criteria  designed  and  developed  for  forces  and 
accelerations  in  the  horizontal  plane  to  address  frontal  impact-induced  injuries)  are  not  appropriate  to  assess  the  effects  of 
the  vertical  forces  and  accelerations  imparted  from  a  combat  UBB  event.  To  address  these  gaps,  I  submitted  an  issue  paper 
in  2010  that  ultimately  provided  $88  Million  for  five  years  of  funding  for  an  Army-led  research  and  development  program  to 
increase  the  Department’s  understanding  of  the  cause  and  nature  of  injuries  incurred  in  UBB  combat  events,  and  to  develop 
appropriate  instrumentation  to  assess  such  injuries  in  testing.  This  project  is  known  as  the  Warrior  Injury  Assessment 
Manikin,  or  WlAMan. 

In  2013,  the  Army  created  a  dedicated  office  (the  WlAMan  Engineering  Office  (WEO))  under  the  Army  Research, 
Development,  and  Engineering  Command  (RDECOM)  to  lead  its  execution  of  the  program.  However,  in  early  2015  the 
office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology  determined  the  WlAMan  project 
would  become  an  Acquisition  Category  II  program  of  record  under  the  Program  Executive  Office  for  Simulation,  Training, 
and  Instrumentation  (PEO  STR1).  Army  PEO  STR1  and  RDECOM  are  developing  a  Test  Capabilities  Requirements 
Document  based  on  the  previous  five  years  of  research  by  the  WEO,  which  I  intend  to  approve  upon  its  completion.  Finally, 
PEO  STRI  worked  with  the  WEO  to  develop  and  validate  a  formal  Program  Office  Estimate  for  full  funding  of  the  program. 
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Unfortunately,  the  Army  elected  not  to  program  any  funding  for  the  WIAMan  project  after  its  initial  five  years  of  funding 
was  to  end  in  FY16,  despite  knowing  the  project  would  not  be  completed  by  then.  This  delay  was,  in  part,  due  to  the 
Army’s  early  mismanagement  of  the  biomechanics  testing,  which  necessitated  restructuring  the  project  in  its  third  year.  This 
restructuring  resulted  in  cost  overruns  and  schedule  delays  that  the  Department  has  not  accounted  for  in  its  allocation  of 
resources  to  WIAMan.  The  Assistant  Secretary  of  Defense  (Health  Affairs)  has  committed  Science  and  Technology  funding 
to  the  program  post-Milestone  B  to  ensure  critical  injury  biomechanics  research  is  completed,  but  this  commitment  has  not 
been  matched  by  a  similar  commitment  from  the  Army  to  program  for  the  anthropomorphic  test  device  (ATD)  production  and 
procurement. 

Some  within  the  Army  question  whether  the  DOD  still  needs  a  combat-specific  injury  assessment  capability  for  UBB  test 
events;  however,  it  is  entirely  appropriate  for  the  DOD,  and  in  particular  for  the  Army,  to  accord  the  same  high  priority 
to  testing  and  verifying  the  protection  provided  to  Soldiers  by  their  combat  vehicles  that  the  commercial  automotive 
industry  accords  to  testing  and  verifying  the  protection  provided  to  the  U.S.  public  by  their  automobiles.  For  example,  the 
U.S.  automotive  industry  has  developed  ATDs  tailored  to  the  multiple  axes  of  impact  that  occur  in  civilian  car  crashes.  This 
includes,  but  is  not  limited  to,  ATDs  to  assess  injuries  from  frontal  impacts,  rear  impacts,  and  side  impacts.  There  is  no  single 
ATD  that  is  acceptable  for  all  automotive  impact  conditions,  even  for  the  relatively  slow  impacts  of  a  car  crash  and  none 
of  these  automotive  ATDs  are  acceptable  for  impact  conditions  observed  in  combat.  The  Army’s  lack  of  a  commitment  to 
completing  this  project  required  me  to  submit  an  issue  paper  this  year  for  additional  funding  of  $98  Million  through  FY21 
that  would  enable  the  continuation  of  development  of  defensible  injury  criteria,  predictive  modeling  and  simulations,  and 
two  generations  of  prototype  ATDs. 

Fifth-Generation  Aerial  Target  (5GAT) 

DOT&E  investigated  the  need  for  an  aerial  target  to  adequately  represent  the  characteristics  of  Fifth  Generation  threat 
aircraft  in  light  of  the  emergence  of  threat  aircraft  like  Russia’s  PAK-FA  and  China’s  J-20.  The  Fifth-Generation  Target 
study  effort  began  in  2006  and  examined  the  design  and  fabrication  of  a  dedicated  5GAT  that  would  be  used  in  the  evaluation 
of  U.S.  weapon  systems  effectiveness.  The  study  team,  comprised  of  Air  Force  and  Navy  experts,  retired  Skunk  Works 
engineers,  and  industry,  completed  a  preliminary  design  review  for  a  government-owned  design.  DOT&E  and  the  TRMC 
have  invested  over  $11  Million  to  mature  the  5GAT  government-owned  design.  Further  investment  is  required  to  complete 
the  prototype.  DOT&E  submitted  an  issue  paper  this  year  for  $27  Million  to  complete  final  design,  tooling,  and  prototyping 
efforts.  The  prototyping  effort  will  provide  cost-informed,  alternative  design  and  manufacturing  approaches  for  future  air 
vehicle  acquisition  programs.  These  data  can  also  be  used  to  assist  with  future  weapon  system  development  decisions,  T&E 
infrastructure  planning/investment,  and  could  support  future  analysis  of  alternative  activities. 

Network  Integration  Evaluation  (NIE) 

In  FY15,  the  Army  executed  two  Network  Integration  Evaluations  (NIEs)  at  Fort  Bliss,  Texas,  and  White  Sands  Missile 
Range,  New  Mexico.  NIE  15.1  was  conducted  in  October  and  November  2014,  and  NIE  15.2  was  conducted  in  April  and 
May  2015.  The  purpose  of  the  NIEs  is  to  provide  a  venue  for  operational  testing  of  Army  acquisition  programs,  with  a 
particular  focus  on  the  integrated  testing  of  tactical  mission  command  networks.  During  NIE  15.1,  the  Army  executed 
an  FOT&E  for  Warfighter  Information  Network  -  Tactical  (WIN-T)  Increment  2.  During  NIE  15.2,  the  Army  conducted 
an  FOT&E  for  the  Distributed  Common  Ground  System  -  Army  (DCGS-A)  and  a  Limited  User  Test  for  the  Mid-Tier 
Networking  Radio  (MNVR).  Individual  articles  on  these  programs  are  provided  elsewhere  in  this  Annual  Report.  Beginning 
in  FY16,  the  Army  will  devote  one  NIE  a  year  to  operational  testing  and  another  annual  event  to  experimentation  and 
force  development.  The  latter  event  is  to  be  called  an  Army  Warfighting  Assessment;  the  first  of  these  was  conducted  in 
October  2015. 

The  Army  Test  and  Evaluation  Command’s  Operational  Test  Command  and  the  Brigade  Modernization  Command,  continue 
to  develop  realistic,  well-designed  operational  scenarios  for  use  during  NIEs.  The  Army  should  continue  to  improve  its 
instrumentation  and  data  collection  procedures  to  support  operational  testing,  including  refining  its  method  for  the  conduct  of 
interviews,  focus  groups,  and  surveys  with  the  units  employing  the  systems  under  test.  The  Army  continues  to  improve  threat 
operations  during  NIEs,  particularly  with  respect  to  threat  information  operations,  such  as  electronic  warfare  and  computer 
network  operations.  NIEs  should  incorporate  a  large,  challenging  regular  force  threat  that  includes  a  sizeable  armored  force 
and  significant  indirect  fire  capabilities. 

Network  components,  both  mission  command  systems  and  elements  of  the  transport  layer,  remain  excessively  complex 
to  use.  The  current  capability  of  an  integrated  network  to  enhance  mission  command  is  diminished  due  to  pervasive  task 
complexity.  It  is  challenging  to  achieve  and  maintain  user  proficiency.  While  networked  communications  at  lower  tactical 
levels  may  create  enhanced  operational  capability,  the  use  of  these  networking  waveforms  brings  negative  attributes,  which 
need  to  be  fully  evaluated  and  understood.  The  challenge  of  integrating  network  components  into  tracked  combat  vehicles 
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remains  unresolved.  Due  to  vehicle  space  and  power  constraints,  the  Army  has  yet  to  successfully  integrate  desired  network 
capabilities  into  Abrams  tanks  and  Bradley  infantry  fighting  vehicles.  It  is  not  clear  how  the  desired  tactical  network  will  be 
incorporated  into  heavy  brigades.  The  WIN-T  FOT&E  conducted  during  N1E  15.1  revealed  significant  problems  with  the 
integration  of  WIN-T  into  Stryker  vehicles.  Integration  of  the  tactical  network  into  an  Infantry  Brigade  Combat  Team  has 
not  been  evaluated  at  NIEs  due  to  the  lack  of  a  light  infantry  unit  assigned  to  the  N1E  test  unit.  Integration  of  the  network 
into  the  light  forces  will  be  challenging  given  the  limited  number  of  vehicles  in  the  Infantry  Brigade  Combat  Team.  The 
intended  tactical  network  places  a  greater  demand  upon  the  available  electromagnetic  spectrum  than  has  been  the  case  with 
non-networked  communications.  An  integrated  tactical  network  introduces  new  vulnerabilities  to  threat  countermeasures, 
such  as  threat  computer  network  attacks,  and  the  ability  of  a  threat  to  covertly  track  friendly  operations.  The  Army  has 
yet  to  integrate  radios  into  its  rotary-winged  aircraft,  which  are  capable  of  operating  in  the  same  network  as  ground  forces 
at  the  company  level  and  below.  Units  remain  overly  dependent  upon  civilian  Field  Service  Representatives  to  establish 
and  maintain  the  integrated  network.  This  dependency  corresponds  directly  to  the  excessive  complexity  of  use  of  network 
components. 

Ballistic  Missile  Defense 

The  Ballistic  Missile  Defense  System  (BMDS)  is  a  system  of  sensors  and  weapons  that  have  not  yet  demonstrated  an 
integrated  functionality  for  efficient  and  effective  defense.  Currently,  the  BMDS  relies  on  man-in-the-loop  processes  to 
integrate  across  the  blue  force  instantiations  for  mission  execution  coordination  within  each  Combatant  Command  because 
the  Command  and  Control,  Battle  Management,  and  Communications  (C2BMC)  element  does  not  provide  engagement 
management  capability  to  the  BMDS.  The  Missile  Defense  Agency  (MDA)  should  continue  C2BMC  development  efforts  to 
provide  an  engagement  management  capability  to  the  BMDS. 

In  its  ongoing  efforts  to  demonstrate  BMD  theater  defense,  the  MDA  conducted  several  system-  and  weapon-level  flight 
and  ground  tests  in  FY/CY15  using  Aegis  Ballistic  Missile  Defense  (Aegis  BMD),  Terminal  High-Altitude  Area  Defense 
(THAAD),  and  Patriot.  However,  the  MDA  still  needs  to  prioritize  development  and  funding  for  a  BMDS  simulation-based 
performance  assessment  capability  including  M&S  validation,  verification,  and  accreditation  and  the  ability  to  produce 
high-fidelity  and  statistically-significant  BMDS-level  performance  assessments.  Aegis  BMD  has  demonstrated  the  capability 
to  intercept  short-  and  medium-range  ballistic  missiles  with  Standard  Missile-3  (SM-3)  Block  IB  interceptors,  but  the 
reliability  of  that  interceptor  needs  to  be  improved.  A  key  component  of  the  MDA’s  efforts  to  improve  SM-3  Block  IB 
reliability  is  the  redesign  of  that  interceptor’s  third-stage  rocket  motor  aft  nozzle  system,  which  must  be  sufficiently 
ground  and  flight  tested  to  prove  its  efficacy.  DOT&E  recommends  that  a  flight  test  of  the  THAAD  system  against  an 
intermediate-range  target  should  occur  as  soon  as  possible.  The  first  THAAD  flight  test  against  an  intermediate-range 
ballistic  missile  (the  expected  threat  class  for  defense  of  Guam  where  THAAD  is  currently  deployed)  was  scheduled  for 
2015,  but  was  delayed  because  of  problems  with  other  BMDS  test  events. 

At  the  BMD  strategic  defense  level,  the  MDA  did  not  conduct  a  Ground-based  Midcourse  Defense  (GMD)  interceptor 
flight  test  in  FY/CY15.  To  improve  and  demonstrate  the  capability  of  the  GMD  and  the  reliability  and  availability  of 
the  operational  Ground-Based  Interceptors  (GBIs),  the  MDA  should  continue  diligently  extending  the  principles  and 
recommendations  contained  in  the  Independent  Expert  Panel  assessment  report  on  the  GB1  fleet  to  all  components  of  the 
BMDS  instantiation  for  Homeland  Defense  and  should  continue  with  their  plans  to  retest  the  Capability  Enhancement-I 
Exo-atmospheric  Kill  Vehicle  in  4QFY17  to  accomplish  the  test  objectives  from  the  failed  Flight  Test  GB1-07  (FTG-07) 
mission.  In  addition,  DOT&E  recommends  that  the  MDA  should  also  determine  additional  sensor  capability  requirements 
for  a  robust  Defense  of  Hawaii  capability. 

Combat  Data 

Combat  operations  over  the  past  14  years  have  resulted  in  a  large  number  of  rotary-wing  aircraft  hit  by  enemy  fire  resulting 
in  aircraft  losses  and  personnel  casualties  (fatalities  and  injuries).  In  2009,  Congress  directed  the  DOD  to  conduct  a  study 
on  rotorcraft  survivability  with  the  specific  intent  of  identifying  key  technologies  that  could  help  reduce  rotary-wing  losses 
and  fatalities.  However,  since  non-hostile  and  non-combat  mishaps  accounted  for  more  than  80  percent  of  the  losses  and 
70  percent  of  the  fatalities,  conclusions  from  the  2009  study  were  concentrated  towards  preventing  mishaps  rather  than 
surviving  direct  combat  engagements.  Since  then,  DOT&E  has  continued  to  analyze  combat  damage  to  rotary-wing, 
fixed-wing,  and  unmanned  aircraft  to  provide  insight  on  the  threats  (including  small  arms,  Man-Portable  Air  Defense 
Systems,  and  rocket-propelled  grenades),  aircraft  components  and  systems,  and  operational  conditions  that  led  to  the  loss  or 
damage  of  aircraft  and  personnel  casualties.  Additionally,  analyses  of  combat-damaged  aircraft  have  been  compared  to  live 
fire  testing  to  determine  if  any  changes  need  to  be  made  in  how  live  fire  test  programs  are  conducted. 
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This  year,  analyses  of  combat  data  have  been  conducted  for  aircrew  currently  engaged  in  combat  operations.  Forward 
deployed  aircrews  have  found  the  data  extremely  valuable  and  have  modified  their  tactics,  techniques,  and  procedures  based 
on  these  data.  Because  combat  data  are  perishable  if  not  collected  immediately,  I  made  a  recommendation  to  institutionalize 
an  Air  Combat  Damage  Reporting  (ACDR)  process  across  the  Department.  Institutionalizing  the  ACDR  will  improve  the 
Department’s  ability  to  understand  and  rapidly  respond  to  changes  in  enemy  tactics  and  weapons  resulting  in  fewer  losses 
of  personnel  and  aircraft  through  mitigation  actions.  The  value  of  the  data  provided  by  ACDR  is  lessened  due  to  a  lack  of 
structured  and  enforced  combat  damage  data  collection  in  theater.  Integrating  ACDR  into  operational  scenario  planning  and 
subsequent  manning  to  deploy  combat  damage  assessment  teams  as  an  integrated  element  of  the  aviation  fighting  force  will 
fill  the  capability  gap.  It  will  enable  the  timely  collection  of  perishable  combat  damage  data  to  support  the  Department’s 
rapid  and  long-term  response  to  current  and  evolving  threats. 

Full  Ship  Shock  Trial  (FSST) 

In  combat,  even  momentary  interruptions  of  critical  systems  can  be  catastrophic  when  those  systems  are  crucial  to  defending 
against  incoming  threats.  This  is  why  the  Navy  has  historically  required  mission-essential  systems  to  remain  functional 
before,  during,  and  after  shock.  The  Navy’s  shock  qualification  specification  states  that  a  momentary  malfunction  is 
acceptable  only  if  it  is  automatically  self-correcting  and  only  if  no  consequent  derangement,  mal-operation,  or  compromise  of 
mission  essential  capability  is  caused  by  the  momentary  malfunction.  The  FSST  will  provide  critical  information  regarding  a 
ship’s  ability  to  survive  and  continue  to  conduct  combat  operations  after  absorbing  hits  from  enemy  weapons.  Understanding 
these  vulnerabilities  is  essential.  Discoveries  made  by  conducting  the  FSST  on  the  first-of-class  ships  will  enable  timely 
modification  of  future  ships  of  the  class  to  assure  their  survivability. 

At  the  direction  of  the  Deputy  Secretary  of  Defense,  the  Navy  is  planning  to  conduct  an  FSST  on  CVN  78  before  her  first 
deployment — to  do  otherwise  would  have  put  CVN  78  at  risk  in  combat  operations.  Flistorically,  FSSTs  for  each  ship  class 
have  identified  previously  unknown  mission-critical  failures  that  the  Navy  had  to  address  to  ensure  ships  would  be  survivable 
in  combat.  We  can  expect  that  CVN  78’s  FSST  results  will  have  significant  and  substantial  implications  on  future  carriers  in 
the  Gerald  R.  Ford  class  and  any  subsequent  new  class  of  carriers. 

Shock  trials  are  routinely  conducted  on  first-of-class  ships,  recently  including  PGFl  1,  LCC  19,  DD  963,  CV  59,  LHA  1, 

FFG  7,  DDG  993,  LSD  41,  MCM  1,  LHD  1,  and  MF1C  1.  Flowever,  on  occasion,  various  circumstances  have  caused  some 
shock  trials  not  to  be  conducted  on  the  first-of-class,  with  the  primary  reason  being  to  ensure  testing  is  conducted  on  the  most 
representative  ship  of  the  class.  For  example,  FSSTs  will  not  be  conducted  on  the  first-of-class  LCSs  because  numerous 
significant  design  changes  are  being  incorporated  in  later  ships.  Nonetheless,  the  preference  is  to  perform  the  FSST  on  the 
first-of-class,  so  as  to  identify  and  mitigate  mission-critical  failures  as  soon  as  possible. 

Some  have  argued  component-level  testing  and  M&S  are  sufficient  to  identify  and  correct  shock-related  problems  on 
fully-integrated  ships.  However,  the  mission-critical  failures  occurring  during  every  FSST,  which  are  conducted  at  less  than 
the  design-level  of  shock,  discredit  this  theory.  For  CVN  78,  the  FSST  is  particularly  important  given  the  large  number  of 
critical  systems  that  have  undemonstrated  shock  survivability.  These  systems  include  the  Advanced  Arresting  Gear  (AAG), 
Electromagnetic  Aircraft  Launching  System  (EMALS),  Dual  Band  Radar  (DBR),  the  13.8  kilovolt  Electrical  Generation 
and  Distribution  Systems,  the  Advanced  Weapons  Elevator  (AWE),  a  new  reactor  plant  design,  and  a  new  island  design  and 
location  with  a  unique  shock  environment. 

It  is  noteworthy  that  the  conduct  of  an  FSST  on  CVN  78  prior  to  her  first  deployment  had  been  a  part  of  the  program  of 
record  since  2004;  therefore,  the  Navy  has  had  ample  time  to  plan  for  this  event.  Nonetheless,  a  number  of  claims  have  been 
and  are  being  made  regarding  the  potential  delay  in  CVN  78’s  deployment  caused  by  conducting  the  FSST  prior  to  the  ship’s 
first  deployment.  These  claims  span  months  to  years;  however,  only  the  former  is  consistent  with  the  Navy’s  conduct  of  the 
FSST  on  CVN  71,  USS  Theodore  Roosevelt.  Commissioned  in  October  1986,  CVN  71  was  underway  most  of  January  and 
February  1987  conducting  crew  and  flight  operations  as  part  of  shakedown.  From  March  to  July  1987,  CVN  71  underwent 
a  post-shakedown  availability.  The  month  of  August  was  used  to  prepare  for  the  FSST,  which  was  conducted  during  the 
period  spanning  August  31,  1987,  to  September  21,  1987.  Upon  completing  the  FSST,  CVN  71  returned  to  Norfolk  Naval 
Station  for  a  two-week  period  to  remove  specialized  trial  equipment  and  to  complete  repairs  to  systems  essential  to  flight 
operations.  After  completing  those  mission-critical  repairs,  CVN  71  returned  to  sea  to  conduct  fleet  carrier  qualifications. 
From  November  1987  to  January  1988,  the  ship  underwent  a  restricted  availability  to  complete  all  post-FSST  and  other 
repairs.  CVN  71  was  then  underway  for  most  of  the  remainder  of  1988,  conducting  independent  steaming  exercises  and 
other  activities,  departing  on  its  first  deployment  on  December  30,  1988.  The  effect  of  conducting  the  FSST  on  CVN  71  ’s 
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availability  for  operations  following  the  shock  trial  was  two  weeks  to  conduct  mission-critical  repairs,  and  the  total  time 
required  to  prepare  for,  conduct,  and  recover  fully  from  the  FSST  was  about  five  months,  including  the  restricted  availability. 

Currently,  FSSTs  for  LCS  5  and  6  are  planned  for  2016.  The  inevitable  lessons  we  will  learn  from  these  tests  will  have 
significant  implications  for  LCS  combat  operations,  as  well  as  for  the  construction  of  the  future  frigate,  which  may  be  based 
on  one  of  the  LCS  designs. 

Despite  the  benefits  of  expedited  FSST,  the  Navy  intends  to  delay  FSST  from  DDG  1000  to  DDG  1002 — a  decision  that  I  do 
not  support.  Conducting  FSST  on  DDG  1000  is  critical  to  finding  and  correcting  failures  in  mission  critical  capabilities  prior 
to  her  first  deployment.  I  submitted  an  issue  paper  this  year  to  restore  the  funding  for  this  test. 


1  submit  this  report,  as  required  by  law,  summarizing  the  operational  and  live  fire  test  and  evaluation  activities  of  the 
Department  of  Defense  during  Fiscal  Year  2015. 
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FY1 5  Activity  Summary 


DOT&E  activity  for  FY15  involved  oversight  of  312  programs, 
including  27  Major  Automated  Information  Systems.  Oversight 
activity  begins  with  the  early  acquisition  milestones,  continues 
through  approval  for  full-rate  production,  and,  in  some  instances, 
during  full  production  until  removed  from  the  DOT&E  oversight 
list. 

Our  review  of  test  planning  activities  for  FY15  included  approval 
of  35  Test  and  Evaluation  Master  Plans  (TEMPs);  84  Operational 
Test  Plans;  4  Live  Fire  Test  and  Evaluation  (LFT&E)  Strategies 
and  Management  Plans  (not  included  in  a  TEMP);  and 
disapproval  of  the  following  1  TEMP  and  1  Test  Plan: 

•  CVN  78  Class  Program  Number  1610,  Revision  C  TEMP 

•  Littoral  Combat  Ship  (LCS)  2  with  Mine  Countermeasure 
(MCM)  Mission  Package  (MP)  Technical  Evaluation 
(TECFIEVAL)  Data  Management  and  Analysis  Plan  (DMAP) 

In  FY15,  DOT&E  prepared  for  the  Secretary  of  Defense  and 
Congress:  9  IOT&E  reports,  5  Early  Fielding  Reports,  4  FOT&E 
reports,  3  LFT&E  reports,  1  Operational  Assessment  (OA) 
report,  2  OT&E  reports,  1  special  report,  and  the  Ballistic  Missile 
Defense  program’s  FY14  Annual  Report.  DOT&E  also  prepared 


and  submitted  numerous  reports  to  the  Defense  Acquisition 
Board  (DAB)  principals  for  consideration  in  DAB  deliberations. 
Additional  FY15  DOT&E  reports  that  did  not  go  to  Congress 
included:  9  Cybersecurity  reports,  3  FOT&E  reports,  3  IOT&E 
reports,  4  LFT&E  reports,  6  OA  reports,  and  3  OT&E  reports. 

During  FY15,  DOT&E  met  with  Service  operational  test 
agencies,  program  officials,  private  sector  organizations,  and 
academia;  monitored  test  activities;  and  provided  information  to 
the  DAB  committees  as  well  as  the  DAB  principals,  the  Secretary 
and  Deputy  Secretary  of  Defense,  USD(AT&L),  the  Service 
Secretaries,  and  Congress.  Active  on-site  participation  in,  and 
observation  of,  tests  and  test-related  activities  are  a  primary 
source  of  information  for  DOT&E  evaluations.  In  addition  to 
on-site  participation  and  local  travel  within  the  National  Capital 
Region,  approximately  790  trips  supported  the  DOT&E  mission. 

Security  considerations  preclude  identifying  classified  programs 
in  this  report.  The  objective,  however,  is  to  ensure  operational 
effectiveness  and  suitability  do  not  suffer  due  to  extraordinary 
security  constraints  imposed  on  those  programs. 


TEST  AND  EVALUATION  MASTER  PLANS  /  STRATEGIES  APPROVED  (INCLUDES  LIVE  FIRE  STRATEGY) 


Abrams  Ml  A2  System  Enhancement  Package  Version  3  Engineering 
Change  Proposal  la  (SEPv3  ECP  la) TEMP* 

Multifunctional  Information  Distribution  System  (MIDS)  Joint  Tactical 
Radio  System  (JTRS)  with  Concurrent  Multinetting/Concurrent 
Contention  Receive  (CMN/CCR)  Capability  on  F/A-1 8  and  EA-1 8G  Aircraft 
Annex  K  (Revision  A)  TEMP 

Armored  Multi-Purpose  Vehicle  (AMPV)TEMP  Update,  Change  1 

Army  Logistics  Modernization  Program  (LMP)  Increment  2  for  the 
Milestone  C  Decision  TEMP 

Bradley  Family  of  Vehicles  (BFoV)  ECP  Program  TEMP* 

Combat  Rescue  Helicopter  (CRH)  TEMP* 

Common  Analytical  Laboratory  System  (CALS)  TEMP 
Common  Aviation  Command  and  Control  System  (CAC2S)TEMP 
Common  Infrared  Countermeasure  (CIRCM)  Milestone  BTEMP 
Defense  Healthcare  Management  Systems  Modernization  TEMP 
Distributed  Common  Ground  System  -  Army  (DCGS-A)TEMP 
Dry  Combat  Submersible  (DCS)  TEMP 
Enhanced  Polar  System  (EPS)  TEMP,  Version  3.0 
F/A-1 8E/F  and  EA-1 8G  TEMP 

F-15  Eagle  Passive/ Active  Warning  Survivability  System  (EPAWSS) 
Milestone  A  TEMP 

Family  of  Advanced  Beyond  Line  of  Sight-Terminal  (FAB-T)TEMP 
Infrared  Search  and  Track  (IRST)  System  ACAT  III  TEMP,  Revision  A 


Integrated  Defensive  Electronic  Countermeasures  (IDECM) 
AN/ALQ-214(V)  Software  Improvement  Program  TEMP 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A)  Increment  2 
Milestone  BTEMP,  Version  1.2 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN),  Increment  4 
TEMP 

Integrated  XM25  Counter  DefiladeTarget  Engagement  (CDTE)  Weapon 
System  TEMP 

Joint  Air-to-Ground  Missile  (JAGM)TEMP* 

Joint  Biological  Tactical  Detection  System  (JBTDS),  ACAT  III,  TEMP 
Supporting  Milestone  B 

Long  Range  Strike  Bomber  (LRS-B)  TEMP 

Miniature  Air  Launched  Decoy -Jammer  (MALD-J)  FOT&E  TEMP 

Mobile  Landing  Platform  (MLP)TEMP  Revision  1.0* 

Next  Generation  Diagnostic  System  (NGDS)  Increment  1  TEMP 
Precision  Guidance  Kit  (PGK)  TEMP 

Presidential  Helicopter  Replacement  Program  (VH-92A)  Revision  ATEMP* 
Small  Diameter  Bomb  Increment  II  (SDB  II)  TEMP* 

Space-Based  Infrared  System  (SBIRS)  Enterprise  TEMP 

Spider  M7E1,  Dispensing  Set,  Munition,  Network  Command,  Increment 
1  ATEMP 

Stryker  Family  of  Vehicles  (FoV)  ECP  TEMP* 
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Theater  Medical  Information  Program  -  Joint  (TMIP-J)  Increment  2,  Warfighter  Information  Network -Tactical  (WIN-T)  Increment  3  TEMP 

Release  3  TEMP 


OPERATIONAL  TEST  PLANS  APPROVED 


Aegis  Weapon  System  (AWS)  Baseline  9C  Integrated  Testing  on  Integrated 
Air  and  Missile  Defense  (IAMD)  Destroyer  DMAP 

Air  Forces  Central  Command  (AFCENT)  Information  Assurance 
Assessment  Plan 

Aegis  Weapon  System  (AWS)  Baseline  9A  Air  Defense  Cruiser  IOT&E  Plan 
AGM-154C-1  Joint  Stand-off  Weapon  (JSOW)  FOT&ETest  Plan 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM)  FOT&ETest 
Plan 

Air  Defense  Cruiser  Aegis  Baseline  9A  Cybersecurity  lOT&ETest  Plan 

Air  Force  Distributed  Common  Ground  System  (AF  DCGS)  Geospatial 
Intelligence  Baseline  (GB)  4.X  Upgrade  Force  Development  Evaluation 
(FDE)  Plan 

Air  Force  Distributed  Common  Ground  System  (AF  DCGS)  System  Release 
3.0  Operational  Utility  Evaluation  (OUE)  Plan 

Air  Force  Distributed  Common  Ground  System  (AF  DCGS)  System  Release 
3.0  Operational  Utility  Evaluation  (OUE)  Plan  Deviation 

Air  Operations  Center  (AOC)  Weapons  System  (WS)  Increment  1 0.1 
Recurring  Event  1 3  FDE  Plan 

Air  Operations  Center  (AOC)  Weapon  System  (WS)  Increment  1 0.2  OA 
Plan 

Air  Warfare/Ship  Self  Defense  (AW/SSD)  Enterprise  (ET05  Phase  2)  Ship 
Self  Defense  System  (SSDS)  Mk2  (OT-IIIH  Phase  2)  and  Rolling  Airframe 
Missile  (RAM)  Block  2  (OT-C2  Phase  2)  Combined  Operational  Test  Plan 

Air  Warfare/Ship  Self  Defense  (AW/SSD)  Enterprise  (ET06),  Ship  Self 
Defense  System  (SSDS)  Mk  2  (OT-IIIH)  FOT&E  and  Rolling  Airframe  Missile 
(RAM)  Block  2  (OT-C3)  IOT&E  Combined  Operational  Test  Plan 

Amphibious  Assault  Ship  Replacement  (LHA  (R)  FLT  0)  IOT&E  Test  Plan 

AN/AAQ-24B(V)25  Department  of  the  Navy  Large  Aircraft  Infrared 
Countermeasures  Advanced  Threat  Warning  System  (DoN  LAIRCM) 
FOT&ETest  Plan 

AN/SQQ-89A(V)1 5  Surface  Ship  Undersea  Warfare  (USW)  Combat  System 
Program  IOT&E  Test  Plan,  Change  2 

AN/TPQ-53  Target  Acquisition  Radar  System  Initial  Operational  Test  2  (IOT 
2)  Operational  Test  Agency  (OTA)  Test  Plan 

Assembled  Chemical  Weapons  Alternatives  (ACWA)  Pueblo  Chemical 
Agent-Destruction  Pilot  Plant  (PCAPP)Test  Concept  Plan  Revision  4Test 
and  Evaluation  Plan 

Automated  Biometric  Identification  System  (ABIS)  1 .2  IOT&E,  Phase  2  Test 
Plan 

Ballistic  Missile  Defense  System  (BMDS)  Flight  Test,  Operational-02 
(FTO-02)  Test  Plan 

Ballistic  Missile  Defense  System  (BMDS)  Integrated  Master  Test  Plan  (IMTP) 
v15.1 

Battle  Control  System  -  Fixed  (BCS-F)  Increment  3  Release  3.2.3  FDE  Plan 

CNO  Project  Number  1 61 0,  CVN-78  Gerald  R.  Ford  Class  Nuclear  Aircraft 
Carrier  OT-B4  OATest  Plan 

Coastal  Battlefield  Reconnaissance  and  Analysis  (COBRA)  OATest  Plan 
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Common  Aviation  Command  and  Control  System  (CAC2S)  Data  Fusion 
Developmental  Test  Observation  Plan 

Common  Aviation  Command  and  Control  System  (CAC2S)  use  of  DT-C2 
Developmental  Test  and  Evaluation  Data  for  IOT&E 

Consolidated  Afloat  Networks  and  Enterprise  Services  (CANES)  FOT&ETest 
Plan  and  Cybersecurity  Plan 

Cooperative  Engagement  Capability  (CEC)  Cybersecurity  FOT&E  Test  Plan 
Cooperative  Engagement  Capability  (CEC)  FOT&E  OT-D1 A  Test  Plan 

CVN-75  (USS  Truman)  Command  and  Control  Exercise  (C2X)  Assessment 
Plan 

Defense  Agencies  Initiative  (DAI)  Increment  2,  Release  1  OA  Plan 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS)  IOT&E 
Plan  Deviation 

Defense  Medical  information  Exchange  (DMIX)  Release  2  OA  Plan 

Defense  Readiness  Reporting  System  -  Strategic  (DRRS-S)  Version  4.6 
IOT&E  Plan 

Department  of  Defense  Teleport  Generation  3,  Phase  3  (G3P3)  OA  Plan 

Distributed  Common  Ground  System  -  Army  (DCGS-A)  Increment  1, 
Release  2  FOT&E  Plan 

Distributed  Common  Ground  System  -  Navy  (DCGS-N)  Increment  1,  Block 
2  FOT&E  Test  Plan  with  the  Cybersecurity  Annex 

EA-18G  FOT&ETest  Plan 

F/A-18e/F  System  Configuration  Set  H10  FOT&ETest  Plan 
F-22  Update  5  Operational  Flight  Program  FDE  Plan  Approval 
F-22A  Increment  3.2A  FOT&E  Plan 

Global  Command  and  Control  System  -  Joint  (GCCS-J)  Version  5  OT&E 
Plan 

Guided  Multiple  Launch  Rocket  System  -  Alternative  Warhead 
(GMLRS-AW)  Implementing  the  Tactics,  Techniques,  and  Procedures  (TTP) 
Demonstration  Test  Plan 

Integrated  Defensive  Electronic  Counter  Measures  (IDECM)  Suite  Block  IV 
FOT&ETest  Plan  Deviation 

Joint  Warning  and  Reporting  Network  (JWARN)  Increment  1  FOT&E  3  Plan 

KC-46A  OA-2  Plan 

KC-46A  OA-2  Plan  Deviation 

Key  Management  Infrastructure  (KMI)  Capability  Increment  2  Spiral  2  Spin 
1  Limited  User  Test  (LUT)  Plan 

Light  Armored  Vehicle  Anti-Tank  Modernization  (LAV-ATM)  OATest  Plan 

Littoral  Combat  Ship  (LCS)  2  Surface  Warfare  (SUW)  (DT-A6)  Integrated 
Testing  DMAP 

Littoral  Combat  Ship  (LCS)  2  with  Mine  Countermeasure  (MCM)  Mission 
Package  Cybersecurity  OT&ETest  Plan 

Littoral  Combat  Ship  (LCS)  4  with  Surface  Warfare  (SUW)  Mission  Package 
Increment  2  IOT&E  Plan 
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Littoral  Combat  Ship  (LCS)  4  with  Surface  Warfare  (SUW)  Mission  Package 
Increment  2  DT/IT-B4  Phase  2  and  DT/IT-C4  DMAP 

Logistics  Modernization  Program  (LMP)  IOT&E  Plan 

Ml  09  Family  of  Vehicles  (FoV)  Self  Propelled  Flowitzer  (SPFH)  5A  Ballistic 
Test  OTA  Test  Plan 

Marine  Corps  FI-1  Upgrades  Program  AFH-1 Z/UFH-1 Y  FOT&ETest  Plan 
Marine  MV-22B  OT-IIIK  FOT&ETest  Plan 

MaxxPro  Long  Wheel  Base  (LWB)  LUTin  support  of  the  Mine  Resistant 
Ambush  Protected  (MRAP)  OTA  Test  Plan 

Mid-Tier  Networking  Vehicular  Radio  (MNVR)  LUT  OTA  Test  Plan 
Miniature  Air  Launched  Decoy  -  Jammer  (MALD-J)  Test  Plan 

MQ-1 C  Gray  Eagle  Unmanned  Aircraft  System  (UAS)  Follow-on 
Operational  Test  (FOT)  and  the  One  System  Remote  Video  Terminal 
(OSRVT)  Increment  II  Initial  Operational  Test  (IOT)  OTA  Test  Plan 

MQ-4CTriton  Unmanned  Aircraft  System  (UAS)  OATest  Plan,  (1 731 -OT-B1 ) 
Nett  Warrior  (NW)  IOT&E  (Phase  2)  OTA  Test  Plan 
Next  Generation  Jammer  (NGJ)  Early  OATest  Plan 

P-8A  Increment  2,  ECP-1  Multi-static  Active  Coherent  (MAC)  OT&ETest 
Plan 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA)  Verification  of 
Correction  of  Deficiencies  (VCD)  Test  Plan 

Pueblo  Chemical  Agent  Destruction  Pilot  Plant  (PCAPP)  Explosive 
Destruction  System  (EDS)  Test  and  Evaluation  Plan 

Remote  Minehunting  System  (RMS)  Integrated  Testing  DMAP 


RQ-4  Global  Flawk  Block  40  IOT&E  Test  Plan 

Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  2  IOT&E 
Cybersecurity  Test  Plan 

Surveillance  Towed  Array  Sonar  System  (SURTASS)/Compact  Low 
Frequency  Active  (CLFA)  IOT&E  Test  Plan  Addendum 

Theater  Medical  Information  Program  -  Joint  (TMIP-J)  Increment  2  Release 
3  Multi-Service  OT&E  Plan 

Torpedo  Warning  System  (TWS)  and  Countermeasure  Anti-Torpedo 
Torpedo  (CAT)  Quick  Reaction  Assessment  (QRA)  DMAP 

TRIDENT  II  (D5)  Strategic  Weapons  Systems  Test  and  Evaluation  Plan  and 
Strategy 

U.S.  Africa  Command  Judicious  Response  201 5  Assessment  Plan 

U.S.  Army  Warfighter  201 5-4  Assessment  Plan  and  Addendum 

U.S.  European  Command  Austere  Challenge  201 5  Final  Assessment  Plan 

U.S.  Northern  Command  Vigilant  Shield  201 5  Cybersecurity  and 
Interoperability  Final  Assessment  Plan 

U.S.  Pacific  Command  Pacific  Sentry  201 5-3  Assessment  Plan 

U.S.  Southern  Command  Integrated  Advance  201 5  Assessment  Plan 

U.S.  Special  Operations  Command  Tempest  Wind  201 5  Assessment  Plan 

U.S.  Transportation  Command  Turbo  Challenge  2015  Final  Assessment 
Plan 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2  FOT&E  2 
OTA  Test  Plan 

XM1 156  Precision  Guidance  Kit  (PGK)  OTA  Test  Plan 


LIVE  FIRE  TEST  AND  EVALUATION  STRATEGIES/MANAGEMENT  PLANS  APPROVED 

Combat  Rescue  Flelicopter  (CRH)  LFT&E  Strategy  Javelin  Spiral  2  Missile  LFT&E  Strategy 

Fleavy  EquipmentTransporter  (HET)  Urban  Survivability  Kit  (HUSK)  LFT&E  VFI-92A  Presidential  Flelicopter  Replacement  Program  LFT&E  Strategy 
Strategy 


Activity 
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FY15  REPORTS  TO  CONGRESS 

PROGRAM 

DATE 

Initial  Operational  Test  and  Evaluation  Reports 

Aegis  Ballistic  Missile  Defense  (BMD)  4.0  and  Standard  Missile-3  Block  IB 

December  2014 

Miniature  Air-Launched  Decoy  with  Jammer  (MALD-J) 

March  2015 

Guided  Multiple  Launch  Rocket  System  -  Alternative  Warhead  (GMLRS-AW) 

March  2015 

Nett  Warrior 

May  2015 

DOD  Automated  Biometric  Identification  System  (ABIS)  Version  1 .2 

May  2015 

RQ-21A  Blackjack  Small  Tactical  Unmanned  Aircraft  System  (STUAS) 

June  2015 

Mobile  Landing  Platform  with  Core  Capability  Set  (MLP  w/CCS) 

July  201 5 

Air  Intercept  Missile  -  9X  (AIM-9X)  Block  II 

July  201 5 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS)  Increment  1  Release  3 

August  201 5 

Early  Fielding  Reports 

Air  Intercept  Missile  -  9X  (AIM-9X)  Block  II 

December  2014 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS)  Increment  1  Release  3 

March  2015 

Massive  Ordnance  Penetrator  (MOP)  Enhanced  Threat  Reduction  Phase  2 

April  201 5 

Aegis  Baseline  9A  Cruiser 

July  201 5 

Virginia  Class  SSN  Block  III 

September  201 5 

Follow-on  Operational  Test  and  Evaluation  Reports 

Handheld,  Manpack,  Small  Form  Fit  (HMS)  AN/PRC-1 55  Manpack  Radio  and  Joint  Enterprise  Network  Manager 
(JENM) 

December  201 5 

Lot  4  AH-64E  Apache  Attack  Helicopter 

December  201 5 

Warfighter  Information  Network -Tactical  (WIN-T)  Increment  2 

May  2015 

Joint  High  Speed  Vessel  (JHSV) 

September  201 5 

Live  Fire  Test  and  Evaluation  Reports 

Stryker  Reactive  Armor  Tiles  (SRAT)  II 

November  2014 

Cartridge  7.62  Ball  M80A1 

February  201 5 

HELLFIRE  Romeo  Final  Lethality  Report 

August  201 5 

Operational  Test  and  Evaluation  Reports 

Joint  Battle  Command  -  Platform  (JBC-P) 

January  2015 

Advanced  Extremely  High  Frequency  (AEHF) 

May  2015 

Operational  Assessment  Reports 

F/A-18E/F  Infrared  Search  and  Track  (IRST)  Block  1 

December  2014 

Special  Reports 

Report  on  the  Littoral  Combat  Ship  (LCS)  required  by  Section  1 23  of  H.R.  3979,  National  Defense  Authorization  Act 
(NDAA) 

April  201 5 

Ballistic  Missile  Defense  Reports 

FY14  Assessment  of  the  Ballistic  Missile  Defense  System  (includes  Classified  Appendices  A,  B,  C,  and  D) 

March  2015 
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OTHER  FY1 5  REPORTS  (NOT  SENT  TO  CONGRESS) 

PROGRAM 

DATE 

Cybersecurity  Reports 

U.S.  Army  Warfighter  Exercise  2014-4 

December  2014 

U.S.  Central  Command  Special  Operations  Command  Central  Headquarters  (HQ)  and  Forward  HQ 

February  201 5 

Integrated  Electronic  Health  Record  (iEHR)  Increment  1 

April  201 5 

U.S.  Air  Forces  Central  Command  2015 

May  2015 

U.S.  Special  Operations  Command-Pacific  Tempest  Wind  2014 

May  2015 

North  American  Aerospace  Defense  Command  and  U.S.  Northern  Command  Vigilant  Shield  2015 

July  201 5 

U.S.  Pacific  Fleet  Valiant  Shield  201 4 

July  201 5 

U.S.  Africa  Command  Judicious  Response  201 5 

August  201 5 

U.S.  Southern  Command  Integrated  Advance  2015 

September  201 5 

Follow-on  Operational  Test  and  Evaluation  Reports 

RQ-7BV2  ShadowTactical  Unmanned  Aircraft  System 

December  2014 

Global  Combat  Support  System  -  Marine  Corps  (GCSS-MC)  Increment  1 

February  201 5 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A)  Increment  1 

March  2015 

Initial  Operational  Test  and  Evaluation  Reports 

Global  Command  and  Control  System  -  Maritime  (GCCS-M)  Increment  2  Version  4.1  Group  Level 

January  2015 

QF-1 6  Full-Scale  Aerial  Target  (FSAT) 

January  2015 

Consolidated  Afloat  Network  and  Enterprise  Services  (CANES)  Program,  Unit  Level  Ship 

July  201 5 

Live  Fire  Test  and  Evaluation  Reports 

HELLFIRE  Romeo  Missile  Variant  (AGM-1 14R-9E) 

November  2014 

MaxxPro  DASH  with  Independent  Suspension  System  (ISS)  and  MaxxPro  Survivability  Upgrade  (MSU) 

April  201 5 

Littoral  Combat  Ship  (LCS)  3  Total  Ship  Survivability  Trial  (TSST) 

August  201 5 

Joint  Light  Tactical  Vehicle  (JLTV) 

August  201 5 

Operational  Assessment  Reports 

Key  Management  Infrastructure  (KMI)  Spiral  2,  Spin  1 

November  2014 

Common  Aviation  Command  and  Control  System  (CAC2S)  Phase  2 

February  201 5 

GBU-53/B  Small  Diameter  Bomb,  Increment  2  (SDB  II) 

May  2015 

Family  of  Advanced  Beyond-Line-of-Sight  Terminals  (FAB-T) 

July  201 5 

Defense  Medical  Information  Exchange  (DMIX)  Release  2 

August  201 5 

Joint  Light  Tactical  Vehicle  (JLTV) 

August  201 5 

Operational  Test  and  Evaluation  Reports 

DOD  Teleport  Generation  3,  Phases  1  and  2 

November  2014 

Global  Command  and  Control  System  -  Joint  (GCCS-J)  Global  Release  4.3  Update  1 

December  2014 

Integrated  Electronic  Health  Record  (iEHR)  Increment  1 

February  201 5 
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Program  Oversight 


DOT&E  is  responsible  for  approving  the  adequacy  of  plans  for 
operational  test  and  evaluation  and  for  reporting  the  operational 
test  results  for  all  Major  Defense  Acquisition  Programs  to  the 
Secretary  of  Defense,  USD(AT&L),  Service  Secretaries,  and 
Congress.  For  DOT&E  oversight  purposes,  Major  Defense 
Acquisition  Programs  were  defined  in  the  law  to  mean  those 
programs  meeting  the  criteria  for  reporting  under  Section  2430, 
Title  10,  United  States  Code  (USC)  (Selected  Acquisition  Reports 
(SARs)).  The  law  (sec. 139(a)(2)(B))  also  stipulates  that  DOT&E 
may  designate  any  other  programs  for  the  purpose  of  oversight, 
review,  and  reporting.  With  the  addition  of  such  “non-major” 
programs,  DOT&E  was  responsible  for  oversight  of  a  total  of  312 
acquisition  programs  during  FY15. 

Non-major  programs  are  selected  for  DOT&E  oversight  after 
careful  consideration  of  the  relative  importance  of  the  individual 
program.  In  determining  non-SAR  systems  for  oversight, 
consideration  is  given  to  one  or  more  of  the  following  essential 
elements: 

•  Congress  or  OSD  agencies  have  expressed  a  high-level  of 
interest  in  the  program. 

•  Congress  has  directed  that  DOT&E  assess  or  report  on  the 
program  as  a  condition  for  progress  or  production. 

•  The  program  requires  joint  or  multi-Service  testing  (the  law 
(sec.  139(b)(4))  requires  DOT&E  to  coordinate  “testing 
conducted  jointly  by  more  than  one  military  department  or 
defense  agency”). 

•  The  program  exceeds  or  has  the  potential  to  exceed  the  dollar 
threshold  definition  of  a  major  program  according  to  DOD 
5000.1,  but  does  not  appear  on  the  current  SAR  list  (e.g., 
highly-classified  systems). 


•  The  program  has  a  close  relationship  to  or  is  a  key  component 
of  a  major  program. 

•  The  program  is  an  existing  system  undergoing  major 
modification. 

•  The  program  was  previously  a  SAR  program  and  operational 
testing  is  not  yet  complete. 

This  office  is  also  responsible  for  the  oversight  of  LFT&E 
programs,  in  accordance  with  10  USC  139.  DOD  regulation  uses 
the  term  “covered  system”  to  include  all  categories  of  systems 
or  programs  identified  in  10  USC  2366  as  requiring  LFT&E.  In 
addition,  systems  or  programs  that  do  not  have  acquisition  points 
referenced  in  10  USC  2366,  but  otherwise  meet  the  statutory 
criteria,  are  considered  “covered  systems”  for  the  purpose  of 
DOT&E  oversight. 

A  covered  system,  for  the  purpose  of  oversight  for  LFT&E, 
has  been  determined  by  DOT&E  to  meet  one  or  more  of  the 
following  criteria: 

•  A  major  system,  within  the  meaning  of  that  term  in  10  USC 
2302(5),  that  is: 

-  User-occupied  and  designed  to  provide  some  degree  of 
protection  to  the  system  or  its  occupants  in  combat 

-  A  conventional  munitions  program  or  missile  program 

•  A  conventional  munitions  program  for  which  more  than 
1,000,000  rounds  are  planned  to  be  acquired. 

•  A  modification  to  a  covered  system  that  is  likely  to  affect 
significantly  the  survivability  or  lethality  of  such  a  system. 

DOT&E  was  responsible  for  the  oversight  of  122  LFT&E 
acquisition  programs  during  FY15. 


Programs  Under  DOT&E  Oversight 
Fiscal  Year  2015 

(As  taken  from  the  September  2015  DOT&E  Oversight  List) 


DOD  PROGRAMS 


AC-130J 

BMDS  -  Ballistic  Missile  Defense  System  Program 

CHEM  DEMIL-ACWA  -  Chemical  Demilitarization  Program  -  Assembled 
Chemical  Weapons  Alternatives 

CHEM  DEMIL-CMA  -  Chemical  Demilitarization  (Chem  Demil)  -  Chemical 
Materials  Agency  (Army  Executing  Agent) 

Common  Analytical  Laboratory  System 
Defense  Agency  Initiative  (DAI) 


Defense  Enterprise  Accounting  and  Management  System  -  Increment  1 
(DEAMS-Inc.  1) 

Defense  Medical  Information  Exchange  (DMIX) 

Defense  Readiness  Reporting  System  -  Strategic 

Defense  Security  Assistance  Management  System  (DSAMS)  -  Block  3 

DoD  Healthcare  Management  System  Modernization  (DHMSM) 

EDS  -  Explosive  Destruction  System 

Enterprise  Business  Accountability  System  -  Defense 
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DOD  PROGRAMS  (continued) 


EProcurement 

Global  Combat  Support  System  -  Joint  (GCSS-J) 

Global  Command  &  Control  System  -  Joint  (GCCS-J) 

Joint  Biological  Tactical  Detection  System 
Joint  Chemical  Agent  Detector  (JCAD) 

Joint  Command  and  Control  Capabilities  (JC2C)  [Encompasses  GCCS-FoS 
(GCCS-J,  GCCS-A,  GCCS-M,  TBMCS-FL,  DCAPES,  GCCS-AF,  USMC  JTCW, 
USMCTCO] 

Joint  Information  Environment 
Joint  Light  Tactical  Vehicle  (JLTV) 

Joint  Warning  and  Reporting  Network  (JWARN) 

Key  Management  Infrastructure  (KMI)  Increment  2 
Mid-Tier  Networking  Vehicle  Radio 


milCloud 

Modernized  Intelligence  Database  (MIDB) 

Modernized  Intelligence  Database  (MIDB) 

Multi-Functional  Information  Distribution  System  (includes  integration 
into  USAF  &  USN  aircraft) 

Next  Generation  Chemical  Detector 

Next  Generation  Diagnostic  System  Increment  1  (NGDS  Inc  1 ) 

Public  Key  Infrastructure  (PKI)  Increment  2 
SOCOM  Dry  Combat  Submersible  Medium  (DCSM) 

Teleport,  Generation  III 

Theater  Medical  Information  Program  -  Joint  (TMIP-J)  Block  2 


ARMY  PROGRAMS 


3rd  Generation  Improved  Forward  Looking  Infrared  (3rd  Gen  FLIR) 
ABRAMS  TANK  MODERNIZATION  -  Abrams  Tank  Modernization  (Ml  E3) 
Abrams  Tank  Upgrade  (Ml  A1  SA/M1 A2  SEP) 

Advanced  Field  Artillery  Tactical  Data  System 
Advanced  Field  Artillery  Tactical  Data  System  (AFATDS)  Version  7 
Advanced  Multi-Purpose  (AMP)  1 20  mm  Tank  Round 
AFI-64E  Apache 

Airborne  and  Maritime/Fixed  Site  Joint  Tactical  Radio  System  (AMFJTRS) 
Small  Airborne  Link  16Terminal  (SALT) 

Airborne  and  Maritime/Fixed  Site  Joint  Tactical  Radio  System  (AMFJTRS) 
Small  Airborne  Networking  Radio  (SANR) 

AMF  Airborne  &  Maritime/Fixed  Station 
AN/PRC-1 17G  Radio 
AN/TPQ-53  Radar  System  (Q-53) 

Armed  Aerial  Scout  (previously  named  ARH  Armed  Recon  Helicopter) 
Armored  Multipurpose  Vehicle  (AMPV) 

Armored  Truck  -  Fleavy  Dump  Truck  (FIDT) 

Armored  Truck  -  Fleavy  EquipmentTransporter  (HET) 

Armored  Truck-  Fleavy  Expanded  Mobility  TacticalTruck  (HEMTT) 
Armored  Truck  -  M91 5A5  Line  Flauler 
Armored  Truck  -  M939  General  Purpose  Truck 
Armored  Truck  -  Palletized  Loading  System  (PLS) 

Army  Vertical  Unmanned  Aircraft  System 
Assured  Precision,  Navigation  &  Timing  (Assured  PNT) 

Biometrics  Enabling  Capability  (BEC)  Increment  1 
Biometrics  Enabling  Capability  Increment  0 
Black  HAWK  (UH-60M)  -  Utility  Helicopter  Program 
Bradley  Engineering  Change  Proposal  (ECP)  and  Modernization 

C-1 7  Increase  Gross  Weight  (IGW)  and  reduced  Formation  Spacing 
Requirements  (FSR)  withT-1 1  parachute 

8  Program  Oversight 


CH-47F  -  Cargo  Helicopter 

Common  Infrared  Countermeasures  (CIRCM) 

Common  Operating  Environment 
Common  Remotely  Operated  Weapons  System  III 
Department  of  Defense  Automated  Biometric  Information  System 
Distributed  Common  Ground  System  -  Army  (DCGS-A) 

EXCALIBUR  -  Family  of  Precision,  1 55  mm  Projectiles 

FBCB2  -  Force  XXI  Battle  Command  Brigade  and  Below  Program 

FBCB2  -  Joint  Capability  Release  (FBCB2  -  JCR) 

Fixed-Wing  Utility  Aircraft 

FMTV  -  Family  of  Medium  Tactical  Vehicles 

Gator  Landmine  Replacement  Program  (GLRP) 

General  Fund  Enterprise  Business  System  (GFEBS) 

Global  Combat  Support  System  -  Army  (GCSS-A) 

Guided  Multiple  Launch  Rocket  System  -  Unitary  (GMLRS  Unitary) 

Guided  Multiple  Launch  Rocket  System  -  Alternate  Warhead  (GMLRS  AW) 
HELLFIRE  Romeo 

High  Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV) 

HIMARS  -  High  Mobility  Artillery  Rocket  System 

Identification  Friend  or  Foe  Mark  XIIA  Mode  5  (all  development  and 
integration  programs) 

Improved  Turbine  Engine  Program 

Indirect  Fire  Protection  Capability  Increment  2  -  Intercept 

Integrated  Air  and  Missile  Defense  (IAMD) 

Integrated  Personnel  and  Pay  System  -  Army  (Army  IPPS)  Increment  1 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A)  Increment  2 

Interceptor  Body  Armor 

Javelin  Antitank  Missile  System  -  Medium 

Joint  Air-to-Ground  Missile 
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ARMY  PROGRAMS  (continued) 


Joint  Assault  Bridge 

Joint  Battle  Command  Platform  (JBC-P) 

Joint  Future  Theater  Lift  Concept  (JFTLC) 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 
Joint  Tactical  Networks  (JTN) 

Logistics  Modernization  Program  (LMP) 

Long  Range  Precision  Fires  (LRPF) 

M270A1  Multiple  Launch  Rocket  System  (MLRS) 

M829E4 

Modernized  Expanded  Capacity  Vehicle  (MECV)  -  Survivability  Project 
Modular  Flandgun  System  (MHS) 

MQ-1 C  Unmanned  Aircraft  System  Gray  Eagle 
Near  Real  Time  Identity  Operations 
Nett  Warrior 

One  System  Remote  Video  Terminal 
Paladin/FASSV  Integrated  Management  (PIM) 

PATRIOT  PAC-3  -  Patriot  Advanced  Capability  3  (Missile  only) 

PATRIOT/MEADS  -  Patriot/Medium  Extended  Air  Defense  System 

RQ-1 1 B  Raven  -  Small  Unmanned  Aircraft  System 

RQ-7B  SFIADOW  -Tactical  Unmanned  Aircraft  System 

Soldier  Protection  System 

Spider  XM7  Network  Command  Munition 

STRYKER  ECP  -  STRYKER  Engineering  Change  Proposal 

Stryker  Ml  126  Infantry  Carrier  Vehicle  including  Double  V-Hull  variant 

StrykerM1 127  Reconnaissance  Vehicle 

Stryker  Ml  1 28  Mobile  Gun  System 


Stryker  Ml  1 29  Mortar  Carrier  including  the  Double  V-Hull  variant 
Stryker  Ml  130  Commander's  Vehicle  including  the  Double  V-Hull  Variant 
Stryker  Ml  131  Fire  Support  Vehicle  Including  the  Double  V-Hull  Variant 

Stryker  Ml  1 32  Engineer  Squad  Vehicle  Including  the  Double  V-Hull 
Variant 

Stryker  Ml  133  Medical  Evacuation  Vehicle  Including  the  Double  V-Hull 
Variant 

Stryker  Ml  1 34  ATGM  Vehicle  Including  the  Double  V-Hull  Variant 
Stryker  Ml  135  NBC  Reconnaissance  Vehicle  (NBCRV) 

Tactical  Mission  Command 
Tactical  Radio  System  Manpack 
Tactical  Radio  System  Rifleman  Radio 
UH-60V  Black  HAWK 
UH-72A  Lakota  Light  Utility  Helicopter 

WIN-T  INCREMENT  1  -  Warfighter  Information  Network -Tactical 
Increment  1 

WIN-T  INCREMENT  2  -  Warfighter  Information  Network -Tactical 
Increment  2 

WIN-T  INCREMENT  3  -  Warfighter  Information  Network -Tactical 
Increments 

WIN-T  INCREMENT  4 -Warfighter  Information  Network -Tactical 
Increment  4 

XM1 156  Precision  Guidance  Kit  (PGK) 

XM1158  7.72  mm  Cartridge 

XM25,  Counter  Defilade  Target  Engagement  (CDTE)  System 
XM395  Accelerated  Precision  Mortar  Initiative  (APMI) 


NAVY  PROGRAMS 


Acoustic  Rapid  COTS  Insertion  for  SONAR 
Advanced  Airborne  Sensor 

Advanced  Extremely  High  Frequency  Navy  Multiband  Terminal  Satellite 
Program  (NMT) 

AEGIS  Modernization  (Baseline  Upgrades) 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile 
AH-1Z 

AIM-9X  -  Air-to-Air  Missile  Upgrade  Block  II 
Air  and  Missile  Defense  Radar  (AMDR) 

Air  Warfare  Ship  Self  Defense  Enterprise 

Airborne  Laser  Mine  Detection  System  (AN/AES-1 )  (ALMDS) 

Airborne  Mine  Neutralization  System  (AN/ASW-235)  (AMNS) 

Airborne  Resupply/Logistics  for  Seabasing 
Amphibious  Assault  Vehicle  Upgrade 
Amphibious  Combat  Vehicle  (ACV) 

AN/APR-39  Radar  Warning  Receiver 


AN/AQS-20  Minehunting  Sonar  (all  variants) 

An/BLQ-1 0  Submarine  Electronics  Support  Measures 
AN/SQQ-89A(V)  Integrated  USW  Combat  Systems  Suite 

Assault  Breaching  System  Coastal  Battlefield  Reconnaissance  and  Analysis 
System  Block  I 

Assault  Breaching  System  Coastal  Battlefield  Reconnaissance  and  Analysis 
System  Block  II 

CANES  -  Consolidated  Afloat  Networks  and  Enterprise  Services 
CH-53K-  Heavy  Lift  Replacement  Program 
Close-In  Weapon  System  (CIWS)  including  SEARAM 
COBRA  JUDY  REPLACEMENT  -  Ship-based  radar  system 
Common  Aviation  Command  and  Control  System  (CAC2S) 

Cooperative  Engagement  Capability  (CEC) 

Countermeasure  Anti-Torpedo 

CVN-78  -  GERALD  R.  FORD  CLASS  Nuclear  Aircraft  Carrier 
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NAVY  PROGRAMS  (continued) 


DDG  1000-ZUMWAET CLASS  Destroyer  -  includes  all  supporting  PARMs 
and  the  lethality  of  the  LRLAP  and  30  mm  ammunition 

DDG  51  -ARLEIGH  BURKE  CLASS  Guided  Missile  Destroyer  -  includes  all 
supporting  PARMs 

DDG  51  Flight  III  -ARLEIGH  BURKE  CLASS  Guided  Missile 
Destroyer-  includes  all  supporting  PARMs 

Department  of  Navy  Large  Aircraft  Infrared  Countermeasures  Program 
Distributed  Common  Ground  System  -  Navy  (DCGS-N) 

Distributed  Common  Ground  System  -  Marine  Corps  (DCGS-MC) 

E-2D  Advanced  Hawkeye 
EA-1 8G  -  Airborne  Electronic  Attack 
Electro-Magnetic  Aircraft  Launching  System 
Enhanced  Combat  Helmet 

Enterprise  Air  Surveillance  Radar  (EASR)  (replacement  for  SPS-48  and 
SPS-49  air  surveillance  radars) 

Evolved  Sea  Sparrow  Missile  (ESSM) 

Evolved  Sea  Sparrow  Missile  Block  2 

F/A-1 8E/F  -  SUPER  HORNET  Naval  Strike  Fighter 

Future  Pay  and  Personnel  Management  Solution  (FPPS) 

Global  Combat  Support  System  -  Marine  Corps  (GCSS-MC) 
Ground/AirTask Oriented  Radar  (G/ATOR) 

Identification  Friend  or  Foe  Mark  XIIA  Mode  5  (all  development  and 
integration  programs) 

Infrared  Search  and  Track  System 
Integrated  Defensive  Electronic  Countermeasures 
Joint  and  Allied  Threat  Awareness  System 
Joint  High  Speed  Vessel  (JHSV) 

JOINT  MRAP  -  Joint  Mine  Resistant  Ambush  Protected  Vehicles 
FOV  -  including  SOCOM  vehicles 

Joint  Precision  Approach  and  Landing  System 
Joint  Stand-Off  Weapon  C-1  variant  (JSOW  C-1 ) 

KC-130J 

Landing  Ship  Dock  Replacement  (LX(R)) 

Large  Displacement  Unmanned  Undersea  Vehicle 

LCS  Surface  Warfare  Mission  Package  Increment  3  -  Interim  Surface  to 
Surface  Missile  including  Longbow  Hellfire  Missile  (or  other  candidate 
missiles  and  their  warheads) 

LHA  6  -AMERICA  CLASS  -  Amphibious  Assault  Ship  -  includes  all 
supporting  PARMs 

LHA  8  Amphibious  Assault  Ship  (America  Class  with  well  deck) 

Light  Armored  Vehicle 

Light  Weight  Tow  Torpedo  Countermeasure  (part  of  LCS  ASW  Mission 
Module) 

Littoral  Combat  Ship  (LCS)  -  includes  all  supporting  PARMs,  and  57  mm 
lethality 

Littoral  Combat  Ship  Mission  Modules  including  30  mm 

Littoral  Combat  Ship  Surface-to-Surface  Missile  (follow  on  to  the  interim 
SSM) 


Littoral  Combat  Ship  Variable  Depth  Sonar  (LCS  VDS) 

Logistics  Vehicle  System  Replacement 

LPD  17  -  SAN  ANTONIO  CLASS  -  AmphibiousTransport  Dock 
Ship  -  includes  all  supporting  PARMs  and  30  mm  lethality 

LSD  41/49  Replacement 
Marine  Personnel  Carrier 

Medium  Tactical  Vehicle  Replacement  Program  (USMC)  (MTVR) 

MH-60R  Multi-Mission  Helicopter  Upgrade 
MH-60S  Multi-Mission  Combat  Support  Helicopter 

MK54torpedo/MK-54VLA/MK54  Upgrades  Including  High  Altitude 
ASW  Weapon  Capability  (HAAWC) 

MK-48  CBASS Torpedo  including  all  upgrades 

Mobile  Landing  Platform  (MLP)  Core  Capability  Set  (CCS)  Variant  and  MLP 
Afloat  Forward  Staging  Base  (AFSB)  Variant 

Mobile  User  Objective  System  (MUOS) 

MQ-4C  Triton 

MQ-8  Fire  Scout  Unmanned  Aircraft  System 
Multi-static  Active  Coherent  (MAC)  System  CNO  project  1 758 
Naval  Integrated  Fire  Control  -  Counter  Air  (NIFC-CA)  From  the  Air 
Naval  Integrated  Fire  Control  -  Counter  Air  (NIFC-CA)  From  the  Sea 
Navy  Enterprise  Resource  Planning  (ERP) 

Next  Generation  Jammer 

Offensive  Anti-Surface  Warfare  Increment  1 

Offensive  Anti-Surface  Warfare,  Increment  2 

OHIO  Replacement  Program  (Sea-based  Strategic  Deterrence)  -  including 
all  supporting  PARMs 

OSPREY  MV-22  -  Joint  Advanced  Vertical  Lift  Aircraft 

P-8A  Poseidon  Program 

Remote  Minehunting  System  (RMS) 

Replacement  Oiler 

Rolling  Airframe  Missile  (RAM)  including  RAM  Block  1 A  Helicopter  Aircraft 
Surface  (HAS)  and  RAM  Block  2  Programs 

RQ-21 A  Unmanned  Aircraft  System  (UAS) 

Ship  Self  Defense  System  (SSDS) 

Ship  to  Shore  Connector 

Small  Surface  Combatant  (also  called  the  Frigate  modification  to  the 
Littoral  Combat  Ship  variants)  including  the  Anti-Submarine  and  Surface 
Warfare  component  systems 

SSN  774  VIRGINIA  Class  Submarine 
SSN  784  VIRGINIA  Class  Block  III  Submarine 
Standard  Missile  2  (SM-2)  including  all  mods 
Standard  Missile-6  (SM-6) 

Submarine  Torpedo  Defense  System  (SubTDS)  including 
countermeasures  and  Next  Generation  Countermeasure  System  (NGCM) 

Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  2 
Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  3 
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NAVY  PROGRAMS  (continued) 


Surface  Mine  Countermeasures  Unmanned  Undersea  Vehicle  (also  called 
Knifefish  UUV)  (SMCM  UUV) 

Surveillance  Towed  Array  Sonar  System/Low  Frequency  Active 
(SURTASS/LFA)  including  Compact  LFA  (CLFA) 

Tactical  Tomahawk  upgrade  (includes  changes  to  planning  and  weapon 
control  system) 

Torpedo  Warning  System  (Previously  included  with  Surface  ShipTorpedo 
Defense  System)  including  all  sensors  and  decision  tools 


TRIDENT  II  MISSILE  -  Sea  Launched  Ballistic  Missile 
UH-1Y 

Unmanned  Carrier  Launched  Airborne  Surveillance  and  Strike  System 

Unmanned  Influence  Sweep  System  (UISS)  include  Unmanned  Surface 
Vessel  (USV)  and  Unmanned  Surface  Sweep  System  (US3) 

USMC  MRAP-Cougar 

VFI-92  -  Presidential  Flelicopter  Fleet  Replacement  Program 


AIR  FORCE  PROGRAMS 


20  mm  PGU-28/B  Replacement  Combat  Round 
Advanced  Pilot  Trainer 

AEFIF  -  Advanced  Extremely  High  Frequency  (AEHF)  Satellite  Program 

AFNet  Modernization  capabilities  (Bitlocker,  Data  at  Rest  (DaR),  Situational 
Awareness  Modernization  (SAMP)) 

AFNET  Vulnerability  Management  (AFVM)  -  Assured  Compliance 
Assessment  Solution  (ACAS) 

AIM-1 20  Advanced  Medium-Range  Air-to-Air  Missile 
Air  Force  Distributed  Common  Ground  System  (AF-DCGS) 

Air  Force  Integrated  Personnel  and  Pay  System  (AF-IPPS) 

Air  Force  Organic  Depot  Maintenance,  Repair  and  Overhaul  Initiative 
(MROi) 

Air  Operations  Center -Weapon  System  (AOC-WS)  initiatives  including 
10.0  and  10.1 

Air  Operations  Center -Weapon  System  (AOC-WS)  initiative  10.2 
Airborne  Signals  Intelligence  Payload  (ASIP)  Family  of  Sensors 

Airborne  Warning  and  Control  System  Block  40/45  Computer  and  Display 
Upgrade 

B-2  Defensive  Management  System  Modernization  (DMS) 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  1 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  2 

B61  Mod  12  Life  Extension  Program 

Battle  Control  System  -  Fixed  (BCS-F)  3.2 

C-1 30J  -  HERCULES  Cargo  Aircraft  Program 

Cobra  Judy  Replacement  Mission  Planning  Tool 

Combat  Rescue  Helicopter  (CRH) 

Command  and  Control  Air  Operations  Suite  (C2AOS)/Command 
and  Control  Information  Services  (C2IS)  (Follow-on  to  Theater  Battle 
Management  Core  Systems) 

ECSS  -  Expeditionary  Combat  Support  System 
Enclave  Control  Node  (ECN) 

EPS  -  Enhanced  Polar  System 

F-15  Eagle  Passive  Active  Warning  Survivability  System 
F-22  -  RAPTOR  Advanced  Tactical  Fighter 
F-35  -  Lightning  II  Joint  Strike  Fighter  (JSF)  Program 
FAB-T  -Family  of  beyond  Line-of-Sight  Terminals 
Full  Scale  Aerial  Target 


GBS  -  Global  Broadcast  Service 

Geosynchronous  Space  Situational  Awareness  Program 

GPS  OCX  -  Global  Positioning  Satellite  Next  Generation  Control  Segment 

GPS-IIIA  —  Global  Positioning  Satellite  III 

Hard  Target  Munition 

Identification  Friend  or  Foe  Mark  XIIA  Mode  5  (all  development  and 
integration  programs) 

Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN)  Increment  2 
Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN)  Increment  4 
Joint  Air-to-Surface  Standoff  Missile  Extended  Range 
Joint  Space  Operations  Center  Mission  System  (JMS) 

JSTARS  Recapitalization 
KC-46  -Tanker  Replacement  Program 
Long  Range  Stand  Off  (LRSO)  Weapon 
Long  Range  Strike  Bomber 
Massive  Ordnance  Penetrator  (MOP) 

Military  GPS  User  Equipment  (GPS  MGUE) 

Miniature  Air  Launched  Decoy  -  Jammer  (MALD-J) 

MQ-9  REAPER  -  Unmanned  Aircraft  System 
Multi-Platform  RadarTechnology  Insertion  Program 

NAVSTAR  Global  Positioning  System  (GPS)  (Includes  Satellites,  Control  and 
User  Equipment) 

OSPREY  CV-22  -  Joint  Advanced  Vertical  Lift  Aircraft 
Presidential  Aircraft  Recapitalization 
Presidential  National  Voice  Conferencing 

RQ-4B  Block  30  -  High  Altitude  Endurance  Unmanned  Aircraft  System 

RQ-4B  Block 40  Global  Hawk  -  High  Altitude  Long  Endurance  Unmanned 
Aircraft  System 

SBIRS  HIGH  -  Space-Based  Infrared  System  Program,  High  Component 

SBSS  B10  Follow-on  -  Space-Based  Space  Surveillance  Block  10  Follow-on 

SF  -  Space  Fence 

SIPRNET  Modernization 

Small  Diameter  Bomb,  Increment  II 

Three-Dimensional  Expeditionary  Long-Range  Radar 

Weather  Satellite  Follow-on  (WSF) 

Wide  Area  Surveillance  (WAS)  Program 
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Problem  Discovery  Affecting  OT&E 


Overview 

Operational  testing  of  new  acquisition  programs  frequently  identifies  new  and  significant  problems  missed  in  earlier  program 
development,  but  it  can  also  find  issues  known  prior  to  testing  that  were  unaddressed.  The  latter  category  is  especially  problematic, 
as  delays  in  addressing  these  problems  only  exacerbate  the  cost  and  time  required  to  fix  them.  Since  2011,  my  annual  report  has 
documented  both  categories  of  problems  and  the  extent  to  which  they  exist  in  programs  undergoing  operational  tests.  This  year,  as  in 
previous  years,  examples  of  both  were  present.  Highlighting  each  of  these  types  of  problems  is  valuable,  as  the  different  natures  of 
these  categories  offer  insights  into  the  actions  needed  to  field  weapons  that  work. 

Discovering  problems  during  operational  testing  is  crucial  for  those  problems  to  be  corrected  prior  to  the  deployment  and  use  of 
a  system  in  combat.  In  many  cases,  an  operational  environment  or  user  is  necessary  to  uncover  the  problem,  as  in  the  Follow-on 
Operational  Test  and  Evaluation  (FOT&E)  of  Department  of  the  Navy  Large  Infrared  Countermeasures  (DoN  LAIRCM)  on 
the  CH-53E  helicopter,  where  it  was  found  that  cycling  the  power  to  reset  system  faults  could  put  the  aircrew  at  risk  during  a 
combat  mission  for  an  extended  time.  Although  the  system  technical  manuals  contained  the  times  needed  for  each  activity  in  the 
power-cycling  sequence,  it  required  operationally  realistic  testing  to  reveal  how  the  combination  of  various  times  could  affect  a 
combat  mission. 


Realistic  operational  testing  can  also  identify  the  full  implications  of  problems  seen  during  developmental  testing  for  success  in 
combat.  This  was  true  in  the  case  of  Ship  Self  Defense  System  (SSDS)  MK  2,  in  which  problems  had  been  observed  in  contractor 
testing  prior  to  the  operational  test  event,  but  correcting  these  problems  was  not  considered  a  high  priority  until  operational  test  results 
showed  the  potential  for  these  problems  to  result  in  missed  cruise  missile  engagements. 


The  discussion  below  provides  an  overview  of 
the  problems  discovered  in  FY15  during  analyses 
of  operational  test  events.  Detailed  accounts 
of  the  discovered  problems  can  be  found  in 
corresponding  individual  program  write-ups  in 
this  report.  I  also  list  48  programs  that  presented 
significant  problems  during  early  testing.  If 
left  uncorrected,  these  deficiencies  could  affect 
my  evaluation  of  operational  effectiveness, 
suitability,  or  survivability.  At  the  conclusion 
of  this  section,  I  report  on  the  progress  of  the 
problems  reported  in  my  FY14  Annual  Report. 

The  results  of  problem  discovery  in  FY15  are 
shown  in  Figure  1.  There  were  134  programs 
on  the  DOT&E  oversight  list  that  planned  or 
conducted  operational  testing  between  FY15 
-  FY17.  Of  those,  66  programs  had  a  total  of 
75  operational  tests  this  year  (some  programs 
had  more  than  one  phase  of  testing  this  year). 

About  one-third  (27/75)  of  the  operational  tests 
conducted  this  year  had  no  significant  problem 
discovery,  while  nearly  two-thirds  (48/75)  revealed 
problems  significant  enough  to  adversely  affect  my 
evaluation  of  the  system's  operational  effectiveness, 
suitability,  or  survivability.  Almost  40  percent 
(29/75)  of  these  operational  tests  discovered 
significant  problems  that  were  unknown  prior  to 
operational  testing. 


134 

Programs  with  operational 
tests  in  FY15-17 


27 

Tests  with  no 
significant 
problems 


r 

9 

Tests  found 
only  new 
problems 


66  programs  underwent 
75  operational  tests  in 
FY15. 


48 

Tests  found 
significant  problems 


Tests  discovered  new 
problems  and  observed 
known  problems 


Tests  only 
observed  known 
problems 


FIGURE  1 .  PROGRAMS/OPERATIONAL  TESTS  UNDER  OVERSIGHT  IN  FY1 5 
WITH  IDENTIFIED  PROBLEMS 

(Note:  Programs  with  testing  in  FY1 5  and  upcoming  testing  in  FY1 6-1 7  may  be  counted  more 
than  once  if  there  were  multiple  test  events.  All  counts  exclude  some  classified  and  chemical 

demilitarization  programs.) 
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Figure  2  shows  the  distribution  of 
the  types  of  significant  problems 
found  during  operational  testing 
(effectiveness,  suitability, 
survivability)  according  to  whether 
the  problem  was  known  prior  to  the 
operational  test.  The  majority  of 
the  problems  (including  75  percent 
of  effectiveness  problems)  were 
known  going  into  operational 
testing.  Many  programs  proceeded 
to  operational  testing  with  known 
problems  because  they  planned 
to  address  the  problems  later.  An 
example  of  this  was  the  P-8A 
Increment  2  Engineering  Change 
Proposal  (ECP)  1,  which  did  not 
meet  its  wide-area  Anti-Submarine 
Warfare  search  requirement  in  some 
environments  because  of  known 
limitations  in  its  Multi-static  Active 
Coherent  (MAC)  sonobuoy  system; 
the  Navy  plans  to  improve  P-8A’s 
performance  with  upgrades  fielded  as 
part  of  Increment  2  ECP  2  in  FY17. 


Effectiveness  Suitability  Survivability 

Problem  Type 


Known 

Unknown 


FIGURE  2.  BREAKDOWN  OF  PROBLEMS  BY  TYPE  AND  WHETHER  THEY  WERE  KNOWN  PRIOR  TO 

OPERATIONAL  TESTING 


Often,  the  realistic  environment  of  operational  testing  provided  new  insights  into  problems  even  if  they  were  known  previously. 
Sometimes  the  scope  of  a  problem  was  not  understood  until  the  system  operated  in  a  realistic  operational  test  environment 
against  realistic  threats,  as  in  the  SSDS  example  above.  Another  example  of  this  case  occurred  with  the  electro-optical/infrared 
(EO/1R)  system  installed  on  the  Independence  variant  of  the  Littoral  Combat  Ship  (LCS)  for  57  mm  gun  engagements.  Although 
developmental  testing  revealed  the  significant  update  rate  and  tracking  performance  problems,  the  full  impact  of  these  deficiencies  was 
only  realized  in  the  operational  test  environment,  in  operational  testing,  these  problems,  combined  with  poor  bearing  accuracy  and 
an  unwieldly  operator  interface,  forced  the  crew  to  supplement  the  watch  team  with  a  dedicated  operator  for  the  sensor.  Even  with 
this  unsustainable  manning  arrangement,  the  ship  was  only  able  to  achieve  modest  gun  performance,  expending  excessive  amounts  of 
ammunition  against  a  single  target. 


In  other  cases,  a  problem  was  rediscovered  after  the  program  thought  it  was  fixed,  either  because  a  technical  fix  did  not  perform 
as  expected  or  because  operational  testing  discovered  that  workarounds  for  a  known  problem  were  impractical  or  not  effective  in 
an  operational  environment.  As  an  example,  the  operational  availability  of  the  Twin-Boom  Extensible  Crane  (TBEC)  used  by  the 
Independence  variant  LCS  to  launch  and  recover  watercraft  is  degraded  by  equipment  failures  and  the  crew's  limited  capability  to 
diagnose  problems  and  repair  the  system  when  it  fails.  Without  the  TBEC,  the  LCS  is  unable  to  launch  and  recover  boats  needed  to 
support  Maritime  Security  Operations  or  Special  Operations  Force  missions,  or  the  Remote  Multi-Mission  Vehicle  (RMMV)  needed 
to  conduct  Mine  Countermeasures  (MCM)  missions.  Following  initial  observation  of  TBEC  problems,  the  Program  Office  worked 
with  the  vendor  to  improve  system  operability  and  refined  shipboard  operating  procedures,  which  resulted  in  some  improvement  in 
watercraft  launch  and  recovery  operations.  Flowever,  there  have  been  continuing  problems  with  the  ability  of  the  ship's  crew  and 
Navy  repair  activities  to  diagnose  problems  and  effect  repairs  without  the  assistance  of  the  original  equipment  manufacturer. 


Finally,  in  other  cases,  a  problem  was  thought  to  be  an  isolated  occurrence  until  it  re-occurred  again  in  operational  testing,  as  in 
the  case  of  the  Integrated  Defensive  Electronic  Countermeasures  (IDECM)  program.  Serious  suitability  problems  (cabin  pressure 
problems  and  avionics  cooling  air  “degrades”  were  seen  at  about  20,000  feet  in  altitude)  for  IDECM  on  the  F/A-18C/D  platform  were 
discovered  during  integrated  test,  but  were  thought  to  be  isolated  problems.  Later  FOT&E  re-observed  the  problems  on  three  jets, 
suggesting  that  the  issues  were  widespread. 
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New  problems  discovered  in  operational  testing  tended  to  cluster  into  several  categories.  New  suitability  problems  were  typically 
caused  by  low  reliability  once  placed  in  an  operational  environment  (7/20),  training  and  documentation  issues  (7/20),  or  usability 
problems  that  prevented  operators  from  successfully  using  a  system  (2/20).  Other  suitability  problems  included  logistics  and  software 
deficiencies.  New  effectiveness  problems  primarily  resulted  from  unexpectedly  low  performance  in  an  operational  environment 
or  against  a  stressing  threat.  Survivability  problems  uncovered  in  FY15  operational  tests  were  all  cybersecurity  vulnerabilities 
(5/5),  which  are  harder  to  uncover  in  developmental  testing.  (Cybersecurity  testing  in  operational  testing  consists  of  Cooperative 
Vulnerability  Penetration  Assessments  (CVPAs)  and  Adversarial  Assessments  (AAs).) 


Figure  3  further  breaks  down 
the  number  of  significant 
problems  discovered  per 
operational  test  by  each  of  the 
Services.  Marine  Corps  tests 
are  included  with  Navy;  Special 
Operations  Command  and  the 
Defense  Agencies  are  grouped 
into  a  fourth  category.  In  some 
cases,  outliers  have  distorted 
the  overall  average.  For 
example,  in  Navy  operational 
tests  this  year,  the  LCS  and 
the  SSDS  experienced  many 
significant  problems.  These  are 
documented  in  the  individual 
write-ups  for  these  programs. 

LCS,  in  particular,  revealed  nine 
significant  problems  in  each 
of  the  two  operational  tests  for 
the  Freedom  and  Independence 
variants.  Effectiveness  problems, 
such  as  those  described  above, 
include  surface  warfare  capabilities, 
air  defense  capabilities,  and  basic 
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FIGURE  3.  HISTOGRAM  SHOWING  THE  NUMBER  OF  PROBLEMS  OBSERVED  IN  EACH  PROGRAM,  BY 
SERVICE.  PROGRAMS  WITH  THE  MOST  PROBLEMS  FROM  EACH  SERVICE  ARE  LABELED. 

(Note:  Navy  includes  the  Marine  Corps;  Other  includes  the  U.S.  Special  Operations  Command,  Missile  Defense 
Agency,  Defense  Logistics  Agency,  Defense  Information  Systems  Agency,  and  National  Security  Agency.) 


ship  functions,  such  as  fuel  endurance  and  boat  handling  equipment.  The  18  problems  for  LCS  also  include  significant  suitability 
problems  with  the  reliability  of  such  systems  as  the  propulsion  and  cooling  systems,  as  well  as  survivability  and  cybersecurity 
problems,  the  latter  only  being  counted  once  for  each  ship  variant  despite  the  existence  of  numerous  deficiencies  in  the  architecture  of 
the  shipboard  networks. 


With  the  exception  of  these  outliers,  the  histograms  in  Figure  3  show  that,  in  general,  the  Services  experience  similar  trends  in 
the  number  of  problems  observed  while  conducting  operational  testing.  Fortunately,  few  programs  experienced  large  numbers  of 
problems  in  operational  testing.  It  is  also  noteworthy  that  each  of  the  Services  experienced  tests  with  no  problems  (Air  Force  6/12, 
Army  9/16,  Navy  (and  Marine  Corps)  9/38);  even  in  these  cases,  the  operational  testing  was  essential  to  confirm  that  users  will  be  able 
to  employ  these  systems  in  realistic  conditions  and  not  be  plagued  by  significant  problems. 

Tables  1  and  2  list  the  75  operational  tests  discussed  in  this  year’s  annual  report.  Each  row  provides  the  name  of  the  system  and 
operational  test,  and  indicates  which  categories  of  problems  were  observed;  for  details  on  the  problems  observed,  see  individual 
system  write-ups  in  this  report. 
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TABLE  1 .  OPERATIONAL  TESTS  (OT)  IN  FY1 5  WITH  NO  SIGNIFICANT  PROBLEM  DISCOVERY 

System  Name 

OT  Name 

AIM-1 20  Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM) 

(pg.  315) 

AMRAAM  Basic  Electronic  Protection  Improvement  Program  (EPIP) 

OT 

AMRAAM 

AMRAAM  System  Improvement  Program  (SIP-1)  OT 

Ballistic  Missile  Defense  System  (BMDS)  (pg.  357) 

BMDS  Flight  Test-Operational  (FTOJ-02 

Common  Aviation  Command  and  Control  System  (CAC2S)  (pg.  175) 

CAC2S  Increment  1  OA 

Consolidated  Afloat  Networks  and  Enterprise  Services  (CANES)  (pg.  1 79) 

CANES  Force-Level  FOT&E 

CANES 

CANES  Unit-Level  IOT&E 

Distributed  Common  Ground  System  -  Army  (DCGS-A)  (pg.  1 07) 

DCGS-A  Increment  1  Release  2  FOT&E 

E-2D  Advanced  Hawkeye  (AHE)  (pg.  1 99) 

E-2D  AHE  FOT&E 

EA-1 8G  Growler  (pg.  201 ) 

H10  Software  Configuration  Set  FOT&E 

Global  Combat  Support  System  -  Army  (GCSS-Army)  (pg.  1 09) 

LSVT  201 5 

Guided  Multiple  Launch  Rocket  System  -  Alternate  Warhead 
(GMLRS-AW)  XM30E1  (pg.  113) 

GMLRS-AW  FOT&E 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A)  (pg.  1 1 7) 

IPPS-A  Increment  1  FOT&E 

Joint  Light  Tactical  Vehicle  (JLTV)  (pg.  1 25) 

JLTV  LUT  (an  operational  assessment) 

Light  Armored  Vehicle- Anti-Tank  Modernization  (LAV-ATM)  (pg.  223) 

LAV-ATM  OA 

KC-46A  (pg.  337) 

KC-46A  OA-2 

Massive  Ordnance  Penetrator  (MOP)  (pg.  341 ) 

Enhanced  Threat  Reduction  Phase  2  OA 

Mine  Resistant  Ambush  Protected  Vehicles  (MRAP)  (pg.  1 37) 

Long  Wheel  Base  (LWB)  Ambulance  LUT  (an  operational  assessment) 

Naval  Integrated  Fire  Control  -  Counter  Air  (NIFC-CA)  From  the  Sea  (FTS) 
(pg.  163, 181) 

AWS  9A/CEC/  NIFC-CA  FTS  DT/OT 

Nett  Warrior  (pg.  143) 

Nett  Warrior  IOT&E 

One  System  Remote  Video  Terminal  (OSRVT)  (pg.  145) 

OSRVT  IOT&E 

Precision  Guidance  Kit  (PGK)  (pg.  1 49) 

PGK  IOT&E 

QF-1 6  Full-Scale  Aerial  Target  (FSAT)  (pg.  349) 

QF-1 6  IOT&E 

Standard  Missile-6  (SM-6)  (pg.  299) 

SM-6  Block  1  FOT&E 

DT  -  Developmental  Test  LUT  -  Limited  User  Test 

FOT&E  -  Follow-on  Operational  Test  and  Evaluation  OA  -  Operational  Assessment 

IOT&E  -  Initial  Operational  Test  and  Evaluation  OT  -  Operational  Test 

LSVT-  Lead  Site  Verification  Test 
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TABLE  2.  OPERATIONAL  TESTS  IN  FY1 5  WITH  DISCOVERY  OF  SIGNIFICANT  PROBLEMS 

System  Name 

Operational  Test 

Effectiveness 

Suitability 

Survivability 

AC-1 30J  Ghostrider  (pg.  309) 

AC-1 30J  Block  10  OA 

X 

X 

X 

Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion 
(A-RCI)  for  AN/BQQ-1 0(V)  Sonar  (pg.  1 59) 

A-RCI  Advanced  Processing  Build  201 1  (APB-1 1 )  Phase  2 
FOT&E 

X 

X 

Aegis  Modernization  Program  (pg.  163) 

Aegis  Weapon  System  (AWS)  9A  (Cruiser 
(CG)/Cooperative  Engagement  Capability  (CEC)/Navy 
Integrated  Fire  Control  -  Counter  Air  (NIFC-CA)  DT/OT 

X 

X 

X 

Aegis  Modernization  Program 

AWS  9A  (CG)  OT 

X 

Aegis  Modernization  Program 

AWS  9C  (Destroyer  (DDG)  OT 

X 

X 

AIM-9X  Air-to-Air  Missile  Upgrade  (pg.  169) 

AIM-9X  Block  II  IOT&E 

X 

Air  Force  Distributed  Common  Ground  System 
(AF  DCGS)  (pg.  317) 

Geospatial  Intelligence  (GEOINT)  Baseline  (GB)  4.1  FDE 
Phase  1 

X 

X 

Air  Operations  Center  -  Weapon  System  (AOC-WS) 

(pg.  321) 

AOC-WS  1 0.1  Recurring  Event  (RE)  1 3  OT 

X 

X 

X 

Airborne  Mine  Neutralization  System  (AMNS;  under  LCS 
and  MH-60S)  (pg.  225, 249) 

MH-60S  with  AMNS  Phase  B  OA  and  concurrent  LCS 

Mine  Countermeasures  (MCM)  Mission  Package  (MP)  DT 

X 

AN/SQQ-89A(V)1 5  Integrated  Undersea  Warfare  (USW) 
Combat  System  Suite  (pg.  1 71) 

Advanced  Capability  Build  (ACB)  1 1  Pre-IOT&E 

X 

Q-53  Counterfire  Target  Acquisition  Radar  System 
(pg.  151) 

Q-53  IOT&E(2) 

X 

Ballistic  Missile  Defense  System  (BMDS)  (pg.  357) 

Aegis  BMD4.0  IOT&E 

X 

Countermeasure  Anti-Torpedo  (CAT)  (under  Surface  Ship 
Torpedo  Defense  (SSTD))  (pg.  303) 

SSTD  QRA  aboard  CVN  71 

X 

X 

CV-22  Osprey  (pg.  325) 

Suite  of  Integrated  Radio  Frequency  Countermeasures 
Block  8  FDE 

X 

CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 
(pg.  183) 

CVN  78  0TB4  0A 

X 

X 

Defense  Agencies  Initiative  (DAI)  (pg.  25) 

DAI  Increment  2  Release  1  OA 

Defense  Enterprise  Accounting  and  Management 

System  (DEAMS)  (pg.  327) 

DEAMS  Release  3  IOT&E 

X 

X 

X 

Defense  Medical  Information  Exchange  (DMIX)  (pg.  27) 

DMIX  Release  2  OA 

X 

X 

Department  of  the  Navy  Large  Aircraft  Infrared 
Countermeasures  (DoN  LAIRCM)  (pg.  197) 

DoN  LAIRCM  Advanced  Threat  Warning  (ATW)  FOT&E  on 
CH-53E 

X 

X 

F-22A  Advanced  Tactical  Fighter  (pg.  33 1 ) 

F-22A  Increment  3.2A  FOT&E 

X 

X 

F/A-1 8E/F  Super  Hornet  (pg.  201 ) 

Super  Hornet  FOT&E 

X 

X 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals 
(FAB-T)  (pg.  335) 

FAB-T  OA-1 

X 

Global  Combat  Support  System  -  Marine  Corps 
(GCSS-MC)  (pg.  203) 

GCSS-MC  Release  1.1.1  FOT&E 

X 

X 

Global  Command  and  Control  System  -  Joint  (GCCS-J) 

(pg.  83) 

GCCS-J  Global  Release  4.3  Update  1  OT&E 

X 

Guided  Multiple  Launch  Rocket  System  -  Alternate 
Warhead  (GMLRS-AW)  M30E1  (pg.  1 1 3) 

GMLRS-AW  IOT&E 

X 

H-1  Upgrades  -  U.S.  Marine  Corps  Upgrade  to  AH-1 Z 
Attack  Helicopter  and  UH-1 Y  Utility  Helicopter  (pg.  205) 

H-1  Upgrades  OT-IIIC  FOT&E 

X 

X 

Infrared  Search  and  Track  (IRST)  (pg.  207) 

IRST  OA  1 

X 

Integrated  Defensive  Electronic  Countermeasures 
(IDECM)  (pg.  209) 

IDECM  Block  4  FOT&E 

X 

X 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A) 

(pg.  117) 

IPPS-A  Adversarial  Assessment 

X 

Joint  High  Speed  Vessel  (JHSV) 

(Expeditionary  Fast  Transport)  (pg.  213) 

JHSV  FOT&E 

X 

X 

Joint  Warning  and  Reporting  Network  (JWARN)  (pg.  91 ) 

Navy  FOT&E  for  JWARN  Increment  1 

X 

X 
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TABLE  2.  OPERATIONAL  TESTS  IN  FY1 5  WITH  DISCOVERY  OF  SIGNIFICANT  PROBLEMS  (CONTINUED) 

System  Name 

Operational  Test 

Effectiveness 

Suitability 

Survivability 

Key  Management  Infrastructure  (KMI)  Increment  2 
(pg.  93) 

KMI  Spiral  2  Spin  1  LUT  (an  operational  assessment) 

X 

KMI  Increment  2 

KMI  Spiral  2  Spin  1  LUT  Retest 
(an  operational  assessment) 

X 

KMI  Increment  2 

KMI  Spiral  2  Spin  1  OA 

X 

LHA  6  New  Amphibious  Assault  Ship  (formerly  LHA(R)) 

(pg.  219) 

LHA6IOT&E 

X 

X 

Littoral  Combat  Ship  (LCS)  ( Freedom  Class)  (pg.  225) 

Freedom  Class  LCS  with  Increment  2  Surface  Warfare 
(SUW)  mission  package  OT 

X 

X 

X 

LCS  ( Independence  Class)  (pg.  225) 

Independence  Class  LCS  with  Increment  2  SUW  mission 
package 

X 

X 

X 

MH-60R  Multi-Mission  Helicopter  (pg.  247) 

LAU-61G/A  Digital  Rocket  Launcher  QRA 

X 

Mid-Tier  Networking  Vehicular  Radio  (MNVR)  (pg.  1 35) 

MNVR  and  Joint  Enterprise  Network  Manager  (JENM) 
LUT  (an  operational  assessment) 

X 

X 

Miniature  Air-Launched  Decoy  (MALD)  and 

MALD  -  Jammer  (MALD-J)  (pg.  343) 

MALD-J  FDE 

X 

X 

Mobile  Landing  Platform  (MLP)  Core  Capability  Set  (CCS) 
(ExpeditionaryTransfer  Dock)  and  Afloat  Forward  Staging 
Base  (AFSB)  (Expeditionary  Mobile  Base)  (pg.  255) 

MLP  CCS  IOT&E 

X 

MQ-1 C  Unmanned  Aircraft  System  Gray  Eagle  (pg.  1 45) 

Gray  Eagle  FOT&E 

X 

X 

MV-22  Osprey  (pg.  67) 

MV-22BOT-IIIK  FOT&E 

X 

X 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA) 
(pg.269) 

P-8A  Increment  2  Engineering  Change  Proposal  (ECP) 

1  FOT&E 

X 

Surface  Electronic  Warfare  Improvement  Program  (SEWIP) 
Block  2  (pg.  301) 

SEWIP  Block  2  IOT&E  (Phase  1) 

X 

X 

Ship  Self-Defense  System  (SSDS)  (pg.  287) 

SSDS  FOT&E 

X 

Surveillance  Towed  Array  Sensor  System  (SURTASS)  and 
Compact  Low  Frequency  Active  (CLFA)  Sonar  (pg.  307) 

SURTASS/CLFA  IOT&E 

X 

X 

Torpedo  Warning  System  (TWS) 

(as  part  of  Surface  ShipTorpedo  Defense  (SSTD))  (pg.  303) 

SSTD  QRA  aboard  CVN  71 

X 

SSN  784  Virginia  Class  Block  ill  Submarine  (pg.  295) 

Virginia  Block  III  Early  Fielding  Certification  Event 

X 

X 

Warfighter  Information  Networking  -Tactical  (WIN-T) 

(pg.  155) 

WIN-T  Increment  2  FOT&E  2 

X 

X 

X 

DT  -  Developmental  Test  MOT&E  -  Multi-Service  Operational  Test  and  Evaluation 

FDE  -  Force  Development  Evaluation  OA  -  Operational  Assessment 

FOT&E  -  Follow-on  Operational  Test  and  Evaluation  OT  -  Operational  Test 

IOT&E  -  Initial  Operational  Test  and  Evaluation  OT&E  -  Operational  Test  and  Evaluation 

LUT  -  Limited  UserTest  QRA  -  Quick  Reaction  Assessment 

There  are  108  operational  tests  scheduled  to  begin  in  the  next  two  fiscal  years,  and  I  am  aware  of  significant  problems,  that  if  not 
corrected,  may  adversely  affect  my  evaluation  of  the  system's  effectiveness,  suitability,  or  survivability  in  48  of  these  systems. 
Table  3  lists  the  upcoming  operational  tests  for  systems  discussed  in  this  year's  annual  report  (see  individual  system  write-ups  in 
this  report  for  details  on  the  problems).  Table  4  lists  the  upcoming  operational  tests  for  systems  that  do  not  have  entries  in  this 
year's  report.  For  these  systems,  I  provide  a  brief  description  of  the  problems  below  the  table. 
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TABLE  3.  PROGRAMS  INTHIS  ANNUAL  REPORT  WITH  PROBLEMS  THAT  MAY  ADVERSELY  AFFECT  UPCOMING  OPERATIONALTESTING 

System  Name 

Upcoming  Test 

Effectiveness 

Suitability 

Survivability 

AC-1 30J  Ghostrider  (pg.  309) 

AC-1 30J  Block  10  IOT&E 

X 

X 

X 

Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion  (A-RCI)  for 

AN/BQQ-1 0(V)  Sonar  (pg.  1 59) 

A-RCI  Advanced  Processing  Build  2013  (APB-13)  FOT&E 

X 

X 

Aegis  Modernization  Program  (pg.  163) 

Aegis  Weapon  System  (AWS)  9C  OT  (DDG) 

X 

X 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM)  (pg.  167) 

AARGM  Block  1  Upgrade  FOT&E 

X 

Air  Force  Distributed  Common  Ground  System  (AF  DCGS)  (pg.  31 7) 

Geospatial  Intelligence  (GEOINT)  Baseline  (GB)  4.1  FDE 

Phase  2 

X 

X 

Air  Operations  Center  -  Weapon  System  (AOC-WS)  (pg.  321 ) 

AOC-WS  10.2  OA 

X 

Airborne  Laser  Mine  Detection  System  (ALMDS;  under  LCS  and  MH-60S) 

(pg.  225, 249) 

Combined  MH-60S  with  ALMDS  Block  1  and  LCS  Mine 
Countermeasures  (MCM)  mission  package  Increment  1 

OT&E 

X 

X 

Airborne  Mine  Neutralization  System  (AMNS;  under  LCS  and  MH-60S) 

(pg.  225, 249) 

Combined  MH-60S  with  AMNS  Block  1  and  LCS  MCM 
mission  package  Increment  1  OT&E 

X 

X 

AN/SQQ-89A(V)15  Integrated  Undersea  Warfare  (USW)  Combat  System 
Suite  (pg.  171) 

Continued  Advanced  Capability  Build  (ACB)  1 1  IOT&E 

X 

Ballistic  Missile  Defense  System  (BMDS)  (pg.  357) 

Aegis  BMD  5.0  Capability  Upgrade  (CU)  OT 

X 

CH-53K-  Heavy  Lift  Replacement  Program  (pg.  1 73) 

CH-53KOT-B1  OA 

X 

Countermeasure  Anti-Torpedo  (CAT)  (under  Surface  Ship  Torpedo 

Defense  (SSTD))  (pg.  303) 

Salvo  Capability  QRA 

X 

X 

CV-22  Osprey  (pg.  325) 

Operational  Utility  Evaluation  (OUE)  of  the  CV-22  Tactical 
Software  Suite  20.2.02/20.2.03 

X 

X 

X 

Defense  Enterprise  Accounting  and  Management  System  (DEAMS) 

(pg.327) 

DEAMS  Inc  1  FOT&E 

X 

X 

X 

Defense  Medical  Information  Exchange  (DMIX)  (pg.  27) 

DMIX  Release  3  IOT&E 

X 

X 

Department  of  the  Navy  Large  Aircraft  Infrared  Countermeasures 
(DoN  LAIRCM)  (pg.  1 97) 

DoN  LAIRCM  Advanced  Threat  Warning  (ATW)  QRA  on 
MV-22 

X 

X 

F-22A  Advanced  Tactical  Fighter  (pg.  331 ) 

F-22A  Increment  3.2B  IOT&E 

X 

F/A-1 8E/F  Super  Hornet  (pg.  201 ) 

System  Configuration  Set  HI  2  and  APG-79  upgrade  OT 

X 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T)  (pg.  335) 

FAB-T  IOT&E 

X 

X 

Global  Command  and  Control  System  -  Joint  (GCCS-J)  (pg.  83) 

GCCS-J  6.0  OT&E 

X 

Infrared  Search  and  Track  (IRST)  (pg.  207) 

IRSTS  Block  1  OA  2 

X 

Integrated  Defensive  Electronic  Countermeasures  (IDECM)  (pg.  209) 

IDECM  Software  Improvement  Program  FOT&E 

X 

X 

Joint  Battle  Command  -  Platform  (JBC-P)  (pg.  1 23) 

JBC-P-Log  FOT&E 

X 

X 

X 

Key  Management  Infrastructure  (KMI)  Increment  2  (pg.  93) 

KMI  Spiral  2  Spin  2  LUT  (an  operational  assessment) 

X 

X 

Littoral  Combat  Ship  (LCS)  (pg.  225) 

Independence  variant  OT  with  the  MCM  mission  package 

X 

X 

X 

Mid-Tier  Networking  Vehicular  Radio  (MNVR)  (pg.  1 35) 

MNVR  IOT&E 

X 

X 

Mobile  User  Objective  System  (MUOS)  (pg.  259) 

MUOS  MOT&E  2 

X 

X 

MQ-4C  Triton  Unmanned  Aircraft  System  (UAS)  (pg.  261 ) 

MQ-4C  IOT&E 

X 

MQ-9  Reaper  Armed  UAS  (pg.  345) 

MQ-9  Block  5  FOT&E 

X 

X 

Nett  Warrior  (pg.  143) 

Nett  Warrior  FOT&E 

X 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA)  (pg.  269) 

P-8A  Increment  2  Engineering  Change  Proposal  (ECP)  2 
FOT&E 

X 

Patriot  Advanced  Capability-3  (PAC-3)  (pg.  147) 

Patriot  Post-Deployment  Build-8  (PDB-8)  IOT&E 

X 

X 

X 

Remote  Minehunting  System  (RMS;  also  addressed  in  LCS)  (pg.  273) 

LCS  MCM  mission  package  Increment  1  OT&E  and 
unofficial  concurrent  RMS  OA 

X 

X 

Surface  Electronic  Warfare  Improvement  Program  (SEWIP)  Block  2 
(pg.  301) 

SEWIP  Block  2  IOT&E  (Phase  2) 

X 

X 

Torpedo  Warning  System  (as  part  of  Surface  Ship  Torpedo  Defense  (SSTD)) 
(pg.303) 

Towed  Active  Acoustic  Source  QRA 

X 

SSN  784  Virginia  Class  Block  III  Submarine  (pg.  295) 

Virginia  Block  III  FOT&E 

X 

X 

Warfighter  Information  Networking  -Tactical  (WIN-T)  (pg.  155) 

WIN-T  Increment  2  Network  Management  and 

Cybersecurity  FOT&E 

X 

X 

FDE  -  Force  Development  Evaluation  OA  -  Operational  Assessment 

FOT&E  -  Follow-on  Operational  Test  and  Evaluation  OT  -  Operational  Test 

IOT&E  -  Initial  Operational  Test  and  Evaluation  OT&E  -  Operational  Test  and  Evaluation 

LUT  -  Limited  User  Test  QRA  -  Quick  Reaction  Assessment 

MOT&E  -  Multi-Service  Operational  Test  and  Evaluation 
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TABLE  4.  PROGRAMS  NOT  IN  THIS  ANNUAL  REPORT  WITH  PROBLEMS  THAT  MAY  ADVERSELY  AFFECT  UPCOMING  OPERATION  ALTESTING 

System  Name 

Upcoming  Test 

Effectiveness 

Suitability 

Survivability 

AH-64E 

AH-64E  Lot  6  FOT&E  II 

X 

AN/BLQ-1 0  Submarine  Electronic  Support  System 

Technical  Insertion  14  (TI-14)  FOT&E 

X 

X 

Coastal  Battlefield  Reconnaissance  and  Analysis  (COBRA) 
System  (also  addressed  in  LCS) 

COBRA  Block  1  IOT&E 

X 

DOD  Automated  Biometric  Identification  System  (ABIS) 

DOD  ABIS  1 .2  Adversarial  Assessment 

X 

GPS  Next  Generation  Operational  Control  System  (OCX) 

OCX  Milestone  C  OA 

X 

X 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  (IFF) 

Mode  5  Joint  Operational 

Test  Approach  (JOTA)  3 

X 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A) 
Increment  II 

IPPS-A  Increment  II  Release  2.0  LUT 
(an  operational  assessment) 

X 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted 
Sensor  System  (JLENS) 

JLENS  Combatant  Command  Integration  Assessment 
(CCIA) 

X 

X 

Military  GPS  User  Equipment  (MGUE)  Increment  1 

MGUE OUE 

X 

X 

MK  54  Lightweight  Torpedo 

MK 54 MODI  IOT&E 

X 

X 

XM25  Counter  Defilade  Target  Engagement  System 

XM25  LUT  (an  operational  assessment) 

X 

FOT&E  -  Follow-on  Operational  Test  and  Evaluation 

IOT&E  -  Initial  Operational  Test  and  Evaluation 

LUT -  Limited  UserTest 

OA  -  Operational  Assessment 

OUE  -  Operational  Utility  Evaluation 

AH-64E.  The  AH-64E  is  a  four-bladed,  twin-engine  attack  helicopter.  The  AH-64E  Lot  6  FOT&E  II  is  scheduled  to  begin 
in  late  FY17. 

Lot  4  AH-64E  and  its  interfacing  systems  have  potentially  significant  cybersecurity  deficiencies.  Further  testing  of  the  AH-64E 
embedded  systems  is  necessary  to  determine  the  significance  of  the  deficiencies. 

AN/BLQ-10.  The  BLQ-10  is  an  electronic  support  system  that  provides  submarines  the  capability  to  detect,  classify,  and 
localize  communications  and  radar  signals.  The  TI-14  FOT&E  is  scheduled  to  begin  in  FY16. 

•  Classified  effectiveness  problems 

•  TI-08  testing  in  FY13  found  that  the  Navy's  training  program  and  promotion  system  does  not  maintain  operator  proficiency  on 
the  communications  subsystem  (of  BLQ-10).  Additionally,  normal  operations  do  not  frequently  involve  the  communications 
subsystem,  so  operators  do  not  have  a  chance  to  maintain  their  proficiency. 

Coastal  Battlefield  Reconnaissance  and  Analysis  (COBRA)  Block  I.  The  COBRA  Block  I  system  is  designed  to  detect  and 
localize  surface  minelines,  minefields,  and  obstacles  in  the  beach  zone  in  support  of  a  beach  landing  by  offensive  forces. 
The  COBRA  Block  I  IOT&E  is  scheduled  to  begin  in  FY16. 

•  During  dynamic  conditions,  such  as  roll  or  pitch  maneuvers,  the  Integrated  Gimbal  (IG)  was  unable  to  maintain  the  correct 
step-stare  sequence.  During  flight  operations,  the  IG  must  continually  look  at  a  single  spot  while  several  images  are  taken.  In 
addition,  the  IG  must  also  adjust  its  look  direction  systematically  to  the  next  correct  spot  to  optimize  its  imagery  acquisition. 
This  process  of  adjusting  the  look  angle  of  the  IG  is  called  the  step-stare  sequence.  Failures  in  the  system  to  maintain  the 
correct  step-stare  sequence  can  result  in  lack  of  imagery  data  for  portions  of  the  target  area  needed  for  Post  Mission  Analysis 
(PMA). 

DOD  Automated  Biometric  Identification  System  (ABIS).  DOD  ABIS  consists  of  information  technology  components 
and  biometric  examiner  experts  that  receive,  process,  and  store  biometrics  from  collection  assets  across  the  globe,  match 
new  biometrics  against  previously  stored  assets,  and  update  stored  records  with  new  biometrics  and  contextual  data  to 
positively  identify  and  verify  actual  or  potential  adversaries.  The  DOD  ABIS  1.2  Adversarial  Assessment  is  scheduled  to 
begin  in  FY16. 

•  There  are  numerous  (classified)  deficiencies,  as  well  as  the  network  defenders'  lack  of  knowledge  of  the  network  architecture, 
that  could  prevent  the  system  from  being  adequately  defended. 


20  Problem  Discovery  Affecting  OT&E 


F  Y  1  5  DOT&E  ACTIVITY  AND  OVERSIGHT 


Global  Positioning  System  (GPS)  Next  Generation  Operational  Control  Segment  (OCX).  GPS  OCX  will  provide 
command  and  control  of  the  GPS  satellite  constellation  and  functions  including  monitoring  and  correction  of  position 
and  time  signals  from  each  satellite,  use  of  modernized  GPS  signals,  and  features  that  support  navigation  warfare 
requirements. 

•  The  Air  Force  has  stated  that  it  needs  to  delay  the  start  of  OCX  operations  from  2018  to  2022  due  to  severe  problems  with 
software  development.  The  Air  Force  has  also  stated  that  delaying  OCX  until  2022  poses  a  significant  risk  of  a  gap  in  GPS 
coverage  starting  in  2019  because  the  Air  Force  requires  OCX  to  operate  the  GPS  III  satellites  the  Air  Force  is  building  to 
sustain  the  GPS  constellation. 

•  To  avoid  a  worldwide  degradation  in  GPS-based  military  and  civilian  positioning,  navigation,  and  timing,  the  Air  Force 
should  prioritize  acquisition  of  a  GPS  III  ground  station  capability  which  can  be  operationally  tested  and  employed  prior  to 
the  constellation  sustainment  need  date  in  2019. 

•  GPS  monitoring  stations  are  inadequate  for  testing  and  operations  of  modernized  GPS  signals,  which  will  prevent  collection 
of  worldwide  signal  quality  data  and  full  evaluation  of  required  navigation  warfare  capabilities  during  both  developmental 
and  operational  testing. 

Mark  XIIA  Mode  5  Identification  Friend  or  Foe  (IFF).  The  Mark  XIIA  Mode  5  IFF  is  a  cooperative  identification 
system  that  uses  interrogators  and  transponders  on  host  platforms  to  send,  receive,  and  process  friendly  identification 
information.  The  Mode  5  Joint  Operational  Test  Approach  (JOTA)  3  is  scheduled  to  begin  in  FY17. 

•  The  system  does  not  meet  the  criteria  for  Lethal  Interrogation  performance.  If  uncorrected,  this  could  result  in  fratricide 
incidents  during  real  world  combat  operations,  especially  in  dense  target  environments. 

•  Identification  information  from  some  Mode  5-equipped  command  and  control  systems  could  not  be  directly  passed  into  the 
command  and  control  system,  limiting  the  ability  of  that  system  to  develop  an  unambiguous  picture  of  the  dynamic  ongoing 
air  battle. 

Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A)  Increment  II.  IPPS-A  is  a  human  resource  system  that 
will  become  the  authoritative  database  for  demographic  information,  deployment  history,  pay,  and  other  personnel 
information  for  the  Army.  The  IPPS-A  Increment  II LUT  is  scheduled  to  begin  in  FY 17. 

•  Personnel  data  in  the  Army  and  DOD  systems  that  interface  with  IPPS-A  need  to  be  verified  as  correct  in  order  for  IPPS-A  to 
provide  accurate  reports 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System  (JLENS).  JLENS  consists  of  separate 
surveillance  and  fire  control  radar  systems  that  are  individually  mounted  on  74-meter  tethered  aerostat  balloons  that 
operate  at  altitudes  up  to  10,000  feet  above  mean  sea  level.  The  JLENS  Combatant  Command  Integration  Assessment 
(CCIA)  is  scheduled  to  begin  in  FY16. 

•  System-level  reliability,  both  software  and  hardware,  is  not  meeting  the  program’s  goals  for  reliability  growth. 

•  Electronic  interference  has  limited  the  surveillance  radar  system  to  certain  frequencies. 

•  The  JLENS  surveillance  radar,  as  initially  configured,  had  certain  features  incorporated  into  its  software  system  intended  to 
deal  with  the  very  high  target  densities  that  exist.  However,  the  design  approach  chosen  to  deal  with  this  problem  resulted  in 
certain  target  sets  being  excluded  by  the  software  algorithms  associated  with  the  surveillance  radar.  This  could  result  in  some 
high  priority  radar  targets  not  being  processed  and  tracked. 

•  Early  testing  has  revealed  problems  related  to  the  timely  passing  of  unambiguous  radar  target  track  infonnation  from  the 
JLENS  system  into  the  North  American  Aerospace  Defense  Command. 

•  During  preparations  for  the  CCIA  exercise,  one  of  the  aerostats  suffered  a  tether  failure  and  was  badly  damaged.  The  accident 
is  under  investigation  and  corrective  actions  will  determine  the  future  direction  of  the  CCIA. 

Military  GPS  User  Equipment  (MGUE)  Increment  1.  MGUE  consists  of  GPS  receivers,  capable  of  receiving  and 
processing  the  new  military  GPS  code  (M-code),  for  all  DOD  platforms  except  precision  guided  munitions,  handheld 
devices,  and  space  vehicles.  M-code  is  designed  to  provide  a  more  secure  and  electronic  warfare  resistant  signal.  The 
MGUE  Operational  Utility  Evaluation  is  scheduled  to  begin  in  FY17. 

•  Developmental  testing  observed  emerging  power  consumption,  peak  power  draw,  thermal  output,  and  messaging  problems  in 
very  early  platform  integration  efforts.  These  problems  might  make  MGUE  incompatible  with  many  DOD  platforms,  driving 
host  platform  and  interface  redesigns  before  those  platforms  can  incorporate  MGUE  and  employ  M-code.  Although  MGUE 
is  expected  to  eventually  integrate  in  nearly  all  platforms  and  munitions  across  the  DOD  portfolio,  the  current  acquisition 
strategy  does  not  involve  significant  integration  testing  on  many  platforms  until  after  completion  of  OT&E  on  the  four  lead 
platforms  in  2019.  Without  wider  testing,  there  is  a  significant  risk  of  late  discovery  of  compatibility  problems  that  could 
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delay  fielding  and  result  in  significant  additional  costs  for  either  re-designing  the  MGUE  itself  or  the  platforms  on  which  the 
MGUE  is  planned  to  be  used. 

•  Messaging  incompatibilities  between  current  MGUE  designs,  existing  platforms,  and  munitions  may  prevent  some  platforms 
from  employing  GPS  M-code  as  required. 

MK  54.  The  MK  54  lightweight  torpedo  is  the  primary  Anti-Submarine  Warfare  weapon  used  by  U.S.  surface  ships, 

fixed-wing  aircraft,  and  helicopters.  The  MK  54  MOD  1  IOT&E  is  scheduled  to  begin  in  FY17. 

•  During  FOT&E,  the  MK  54  MOD  0  demonstrated  below  threshold  performance  in  many  scenarios. 

•  Launch  platforms  are  not  always  able  to  launch  the  MK  54  torpedo  in  a  manner  that  can  support  an  effective  attack. 

•  The  MK  54  torpedo  does  not  always  interface  properly  with  the  fire  control  systems  on  its  launch  platforms. 

•  There  are  several  other  classified  problems. 

XM25  Counter  Defilade  Target  Engagement  System  (CDTE).  The  XM25  CDTE  fires  25  mm  programmable  high- 

explosive  airburst  rounds  to  defeat  defilade  and  point  area  targets  out  to  500  meters.  A  Limited  User  Test  is  scheduled  to 

begin  in  FY16. 

•  The  Army  conducted  three  Forward  Operational  Assessments  of  the  XM25  CDTE  with  prototype  weapons  in  2011,  2012, 
and  2013,  each  of  which  resulted  in  a  weapon  malfunction  and  minor  injuries  to  the  operators.  The  program  conducted  a 
root  cause  investigation  and  made  design  changes  to  ensure  the  safety  of  the  weapon.  Developmental  testing  of  the  modified 
design  is  ongoing. 


PROGRESS  UPDATES  ON  DISCOVERIES  REPORTED  IN  THE  FY1 4  ANNUAL  REPORT 

In  my  annual  report  last  year,  1  identified  8  systems  that  discovered  new  problems,  10  systems  that  discovered  new  problems  and 
re-observed  known  problems,  and  15  systems  that  re-observed  known  problems  during  operational  testing  in  FY14.  The  status 
of  these  33  programs  is  listed  below. 

All  fixes  implemented  and  verified  in  OT 

•  AIM-120D  Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM) 

•  Joint  Light  Tactical  Vehicle  (JLTV) 

•  QF-16  Full-Scale  Aerial  Target  (FSAT) 

Some  (or  all)  fixes  implemented  but  new  problems  discovered  or  known  problems  re-observed  in  OT 

•  Aegis  Ballistic  Missile  Defense  (Aegis  BMD) 

•  Air  Force  Distributed  Common  Ground  System  (AF  DCGS) 

•  Air  Operations  Center  -  Weapon  System  (AOC-WS) 

•  AN/SQQ-89A(V)15  Integrated  Undersea  Warfare  (USW)  Combat  System  Suite 

•  Ballistic  Missile  Defense  System  (BMDS) 

•  CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 

•  Defense  Enterprise  Accounting  and  Management  System  (DEAMS) 

•  Defense  Medical  Information  Exchange  (DMIX) 

•  F/A-18E/F  Super  Flornet  and  EA-18G  Growler 

•  Infrared  Search  and  Track  (1RST) 

•  Joint  High  Speed  Vessel  (JHSV) 

•  Joint  Warning  and  Reporting  Network  (JWARN) 

•  Littoral  Combat  Ship  (LCS) 

•  Miniature  Air-Launched  Decoy  (MALD)  and  MALD  -  Jammer  (MALD-J) 

•  Multi-Static  Active  Coherent  (MAC)  System 

•  MV-22  Osprey 

•  P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA) 

•  Q-53  Counterfire  Target  Acquisition  Radar  System 

•  Surface  Ship  Torpedo  Defense  (SSTD)  System:  Torpedo  Warning  System  (TWS)  and  Countermeasure  Anti-Torpedo  (CAT) 

•  Surveillance  Towed  Array  Sensor  System  (SURTASS)  and  Compact  Low  Frequency  Active  (CLFA)  Sonar 

Some  fixes  (potentially)  implemented;  currently  in  OT  or  planning  additional  OT 

•  Battle  Control  System  -  Fixed  (BCS-F) 

•  DOD  Automated  Biometric  Identification  System  (ABIS) 
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•  Joint  Battle  Command  -  Platform  (JBC-P) 

•  Manpack  Radio 

•  Mark  XIIA  Mode  5  Identification  Friend  or  Foe  (IFF) 

•  MK  54  Lightweight  Torpedo 

No  fixes  planned,  or  no  fixes  planned  to  be  tested  in  the  next  two  years 

•  AN/PRC- 117G 

•  Distributed  Common  Ground  System  -  Marine  Corps  (DCGS-MC) 

•  F-15E  Radar  Modernization  Program  (RMP) 

•  RQ-21A  Blackjack  (formerly  Small  Tactical  Unmanned  Aerial  System  (STUAS)) 

In  FY14, 1  also  identified  23  systems  that  had  significant  issues  in  early  testing  that  should  be  corrected  before  operational 
testing.  The  following  provides  an  update  on  the  progress  these  systems  made  in  implementing  fixes  to  those  problems. 

Fixes  verified  in  OT  -  No  other  problems  observed 

•  Distributed  Common  Ground  System  -  Army  (DCGS-A) 

•  Key  Management  Infrastructure  (KMI) 

•  Precision  Guidance  Kit  (PGK) 

Fixes  verified  in  OT  -  New  problems  discovered 

•  AN/SQQ-89A(V)15  Integrated  Undersea  Warfare  (USW)  Combat  System  Suite 

Fixes  verified  in  OT  -  Known  problems  re-observed 

•  Ballistic  Missile  Defense  System  (BMDS) 

•  Infrared  Search  and  Track  (IRST) 

•  LHA  6  New  Amphibious  Assault  Ship  (formerly  LHA(R)) 

•  Littoral  Combat  Ship  (LCS) 

•  Mobile  Landing  Platform  (MLP)  Core  Capability  Set  (CCS)  (Expeditionary  Transfer  Dock)  and  Afloat  Forward  Staging  Base 
(AFSB)  (Expeditionary  Mobile  Base) 

Fixes  tested  in  OT  -  Both  new  problems  discovered  and  known  problems  re-observed 

•  AC-130J  Ghostrider 

•  Air  Force  Distributed  Common  Ground  System  (AF  DCGS) 

•  Defense  Enterprise  Accounting  and  Management  System  (DEAMS) 

•  Defense  Medical  Information  Exchange  (DMIX) 

•  Warfighter  Information  Network  -  Tactical  (WIN-T) 

Fixes  not  planned  to  be  tested  in  the  next  two  years 

•  F-35  Joint  Strike  Fighter  (JSF) 

Fixes  currently  being  tested  or  planned  to  be  tested  in  the  next  two  years 

•  M829E4  Armor  Piercing,  Fin  Stabilized,  Discarding  Sabot  -  Tracer  (APFSDS-T) 

•  Public  Key  Infrastructure  (PKI) 

•  RQ-4B  Global  Hawk  High- Altitude  Long-Endurance  Unmanned  Aerial  System  (UAS) 

•  AGM-88E  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 

•  MQ-4C  Triton  Unmanned  Aircraft  System 

•  MQ-9  Reaper  Armed  Unmanned  Aircraft  System  (UAS) 

•  Patriot  Advanced  Capability-3  (PAC-3) 

•  Remote  Minehunting  System  (RMS) 
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Defense  Agencies  Initiative  (DAI) 


Executive  Summary 

•  During  the  operational  assessment  conducted  by  the 
Joint  Interoperability  Test  Command  (J1TC)  from 
July  20  through  August  14,  2015,  the  Defense  Agency 
Initiative  (DAI)  Increment  2  Release  1  critical 
functionality  and  interfaces  worked  as  designed. 

•  The  DAI  system  usability  requires  ongoing 
improvements  for  several  defense  agencies.  The 
conversion  to  Oracle  eBusiness  Suite  Release  12  as 
part  of  DAI  Increment  2  Release  1  has  been  difficult, 
resulting  in  slow  response  times  and  defects  requiring 
software  patches  to  fix. 
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System 

•  DAI  is  an  information  technology  system  with  a  core 
functionality  that  provides  a  commercially-available 
Oracle  Enterprise  Resource  Planning  system. 

•  DAI  modernizes  the  financial  management  processes 
for  many  DOD  agencies  and  field  activities  by 
streamlining  financial  management  capabilities, 
addressing  financial  reporting  material  weaknesses, 
and  supporting  financial  statement  auditability. 

•  The  Defense  Information  Systems  Agency  (DISA)  provides 
facilities  for  the  DAI  servers  at  its  Ogden,  Utah,  and 
Columbus,  Ohio,  Defense  Enterprise  Computing  Centers. 

•  DAI  is  employed  worldwide  and  across  a  variety 
of  operational  environments  via  a  web  portal  on  the 
Non-classified  Internet  Protocol  Network  using  each  agency’s 
existing  information  system  infrastructure. 

•  DAI  includes  two  software  increments: 

-  Increment  1  was  in  Operations  and  Sustainment  and  was 
used  for  financial  reporting  at  12  defense  agencies. 

-  Increment  2  has  four  software  releases,  each  with 
additional  capabilities,  with  deployments  to  1 1  additional 
defense  agencies  continuing  through  FY17. 

•  DAI  supports  financial  management  requirements  in  the 
Federal  Financial  Management  Improvement  Act  and  DOD 
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Business  Enterprise  Architecture;  therefore,  it  is  subject  to  the 
2010  National  Defense  Authorization  Act  requirement  to  be 
auditable  by  2017. 

Mission 

Defense  agencies  use  DAI  for  budget,  finance,  and  accounting 
operations  to  provide  accurate,  reliable,  and  auditable  financial 
information  that  supports  DOD  missions. 

Major  Contractors 

•  CACI  Arlington  -  Northampton  County,  Virginia 

•  International  Business  Machines  -  Armonk,  New  York 

•  Northrop  Grumman  -  Falls  Church,  Virginia 


Activity 

•  JITC  conducted  two  developmental  tests  of  DAI  Increment  2 
Release  1:  a  System  Integration  Test  from  February  9,  2015, 
through  March  25,  2015,  followed  by  a  User  Acceptance  Test 
conducted  from  March  30,  2015,  through  April  17,  2015. 

•  JITC  conducted  an  operational  assessment  of  DAI  Increment  2 
Release  1  from  July  20  through  August  14,  2015,  in 
accordance  with  a  DOT&E-approved  test  plan. 

•  A  DISA  Field  Security  Operations  Red  Team  conducted  a 
cybersecurity  Adversarial  Assessment  of  DAI  Increment  1 
Release  3  from  October  14  through  November  25,  2014. 


•  On  September  16,  2015,  USD(AT&L)  signed  an  Acquisition 
Decision  Memorandum  approving  limited  fielding  of  DAI 
Increment  2  Release  2  to  current  and  additional  defense 
agencies. 

•  The  DAI  Program  Management  Office  (PMO)  has  coordinated 
for  a  full  cybersecurity  test  (Cooperative  Vulnerability  and 
Penetration  Assessment  and  Adversarial  Assessment)  for 
2QFY16  on  Increment  2  Release  2. 
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Assessment 

•  During  the  operational  assessment,  the  system’s  critical 
functionality  and  interfaces  worked  as  designed;  however,  the 
testing  revealed  deficiencies  that  reduced  user  satisfaction. 
Deficiencies  included: 

-  Workflow  and  certification  problems  that  affected 
the  approval  of  some  financial  documents  and  forced 
resubmission. 

-  The  time  to  process  employee  payroll  records  was  a 
queued,  serial  process  that  often  took  hours  to  complete  for 
each  agency.  To  mitigate,  the  DAI  PMO  has  assumed  this 
task  until  an  acceptable  software  solution  is  implemented 
and  tested. 

-  There  were  often  long  system  response  times  and  time 
outs  that  required  users  to  take  extra  steps  to  complete 
their  tasks.  The  DAI  help  desk  effectively  supported  the 
production  system  during  the  operational  test  and  worked 
well  with  D1SA  and  defense  agency  customers. 

•  The  conversion  to  Oracle  eBusiness  Suite  Release  12  was  a 
challenge,  but  the  PMO  employed  effective  patch  management 
to  resolve  many  issues  during  and  after  testing. 


•  System  stability  issues  led  to  seven  critical  failures  between 
May  4  and  July  7,  2015.  To  alleviate  the  failure  rate,  the 
program  manager  rescheduled  weekly  maintenance  to 
weekends  and  turned  off  system  logging  to  reduce  the 
demand  on  the  system.  DAI  meets  operational  availability 
requirements  for  peak  and  non-peak  periods. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY15  Recommendations.  The  DAI  PMO  should: 

1 .  Improve  system  response  times,  reduce  time  outs,  and 
correct  other  errors  requiring  users  to  take  extra  steps  to 
complete  their  tasks. 

2.  Establish  a  scheduled  maintenance  program  that  supports  all 
agencies’  missions  that  accounts  for  the  defense  agencies’ 
worldwide  and  weekend  operations. 

3.  Complete  cybersecurity  testing,  to  include  a  Cyber 
Economic  Vulnerability  Assessment,  at  the  Program  Office, 
data  centers,  and  customer  agencies  during  FY16. 
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Defense  Medical  Information  Exchange  (DMIX) 


Executive  Summary 

•  In  December  2014,  USD(AT&L)  separated  the  Defense 
Medical  Information  System  (DMIX)  into  two  components: 
Interoperability  and  Viewer.  USD(AT&L)  declared  the 
Interoperability  component  to  be  under  sustainment  and 
designated  the  Viewer  component  as  a  tailored  Automated 
Information  System  Acquisition  Category  III  program. 

•  The  United  States  Army  Medical  Department  Board 
(USAMEDDBD)  and  the  Army  Test  and  Evaluation  Command 
(ATEC)  performed  an  operational  assessment  of  DMIX 
Release  2  from  April  20,  2015,  through  May  1,  2015,  to 
support  a  Milestone  C  decision  for  the  Viewer  component 

of  the  DMIX  program.  The  USAMEDDBD  conducted  an 
operational  assessment  of  DMIX  Release  2  because  the 
DMIX  program  manager  will  implement  the  Virtual  Lifetime 
Electronic  Record  (VLER),  a  major  capability  of  the  system, 
in  DMIX  Release  3.  The  USAMEDDBD  plans  to  conduct  a 
Multi-Service  Operational  Test  and  Evaluation  (MOT&E)  of 
DMIX  Release  3  with  the  VLER  capability  and  operational 
commercial  health  care  partners. 

-  Based  on  the  operational  assessment,  DOT&E  determined 
that  the  system  was  operationally  suitable,  but  not 
operationally  effective  because  of  high-severity  defects  that 
included  (1)  inconsistent  query  results  for  which  the  system 
did  not  always  display  all  records  available  in  a  date  range, 
(2)  medical  record  notes  not  displaying  due  to  the  format 
of  the  original  note,  and  (3)  inconsistencies  in  data  displays 
and  navigation  buttons  between  different  display  schemes 
within  the  DMIX  viewer. 

-  Although  the  high-severity  defects  affected  data 
completeness,  all  data  displayed  in  the  Joint  Legacy  Viewer 
(JLV)  were  accurate  and  patient  records  downloaded  in  a 
timely  manner. 

•  The  Program  Management  Office  (PMO)  took  immediate  steps 
to  correct  these  problems.  The  PMO  scheduled  a  Verification 
of  Correction  of  Deficiencies  (VCD)  in  December  2015  to 
verify  DMIX  Release  2  fixes. 

•  After  the  deployment  of  Release  3  in  September,  users 
discovered  two  new  system  defects  and  an  external  system 
data  accessibility  problem.  The  DMIX  program  manager 
stated  that  she  has  developed  a  plan  to  resolve  the  DMIX 
Release  3  system  defects  and  data  accessibility  problem. 

•  Space  and  Naval  Warfare  Systems  Command  (SPAWAR) 
conducted  a  cybersecurity  Cooperative  Vulnerability  and 
Penetration  Assessment  (CVPA)  of  DMIX  Release  2  in  the 
developmental  test  environment  that  concluded  in  June  2015. 
The  CVPA  identified  vulnerabilities  that  could  result  in  the  loss 
of  confidentiality,  integrity,  or  availability  of  personal  health 
information  and  personally  identifiable  information. 

•  SPAWAR  conducted  a  cybersecurity  CVPA  of  Integrated 
Electronic  Health  Record  (iEHR)  Increment  1  from 
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AHLTA- Armed  Forces  Health  Longitudinal  Technology  Application 

BHIE  -  Bidirectional  Health  Information  Exchange 

CHDR  -  Clinical  Data  Repository/Health  Data  Repository 

DES  -  Data  Exchange  Service 

DHA  -  Defense  Health  Agency 

DHMSM  -  Defense  Healthcare  Management  System  Modernization 
DOD  -  Department  of  Defense 
JLV  -  Joint  Legacy  Viewer 
VA  -  Veterans  Affairs 


October  27,  2014,  through  December  10,  2014,  and  an 
adversarial  assessment  from  January  12  -  16,  2015.  The 
CVPA  of  iEHR  Increment  1  identified  vulnerabilities  that 
could  compromise  medical  information  if  not  quickly  and 
adequately  addressed. 

System 

•  The  DMIX  program  supports  integrated  sharing  of 
standardized  health  data  among  DOD’s  Defense  Healthcare 
Management  System  Modernization  program,  DOD  legacy 
systems,  Veterans’ Affairs  (VA),  other  Federal  agencies,  and 
private-sector  healthcare  providers. 

•  Together,  the  Defense  Healthcare  Management  System 
Modernization  and  DMIX  are  intended  to  modernize  the 
Military  Health  System  to  enhance  sustainability,  flexibility, 
and  interoperability  for  improved  continuity  of  care. 

•  The  DOD  is  developing  DMIX  incrementally,  delivering 
upgrades  to  capabilities  that  have  already  been  fielded: 

-  JLV  provides  an  integrated  read-only,  chronological  view 
of  health  data  from  DOD  and  VA  Electronic  Health  Record 
(EHR)  systems,  eliminating  the  need  for  VA  or  DOD 
clinicians  to  access  separate  viewers  to  obtain  real-time 
patient  information.  DOD  and  VA  users  logon  to  their 
respective  JLV  Web  Servers  using  a  Uniform  Resource 
Locator  (URL)  address  in  their  web  browser.  Armed 
Forces  Health  Longitudinal  Technology  Application  users 
can  connect  to  the  JLV  Web  Server  through  their  system 
menu. 


DMIX  27 


F  Y  1  5  DOD  PROGRAMS 


-  The  Data  Exchange  Service  (DES)  receives  user  queries 
entered  through  JLV  and  queries  hundreds  of  DOD  and  VA 
data  stores,  returning  the  results  to  jMeadows.  jMeadows 
maps  local  VA  and  DOD  clinical  terms  to  standard  medical 
terminology  and  aggregates  the  data  for  presentation  by  the 
JLV  Web  Server. 

-  The  Bidirectional  Health  Information  Exchange  (BH1E) 
enables  the  VA  to  access  clinical  data  from  multiple 
DOD  and  VA  systems  using  the  Data  Exchange  Service, 
BHIE  Share,  and  Clinical  Data  Repository /Health  Data 
Repository.  The  Clinical  Data  Repository/Health  Data 
Repository  enables  bidirectional  exchange  of  outpatient 
pharmacy  and  medication  allergy  data  for  checking 
drug-to-drug  and  drug-to-allergy  interactions. 

-  The  VLER  capability  provides  views  of  a  patient’s  medical 
history  and  outpatient  clinical  visits  within  DOD  medical 
facilities.  The  VLER  provides  the  ability  to  both  retrieve 
and  share  medical  documentation  with  external  partners, 
such  as  the  VA  and  other  Federal  or  commercial  health 
care  partners.  The  DOD  and  external  partners  pass  VLER 
data  through  the  eHealth  Exchange  service.  The  DMIX 
program  manager  implemented  VLER  viewer  functionality 
into  JLV,  replacing  the  legacy  VLER  viewer. 

•  iEHR,  not  shown  in  the  system  diagram,  is  a  program 
deployed  to  a  single  site,  the  James  A.  Lovell  Federal 
Healthcare  Center  in  North  Chicago,  Illinois,  where  it  is 
now  in  sustainment.  iEHR  provides  single  sign-on  and 


context  management  capabilities  to  enable  a  user  to  logon 
to  all  published  applications  via  a  Common  Access  Card.  It 
allows  users  to  enter  a  patient  once  and  the  same  patient  will 
automatically  populate  in  other  applications.  iEHR  also 
provides  a  roaming  capability  to  allow  users  to  access  their 
information  from  multiple  devices. 

Mission 

The  DOD,  VA,  Federal  agencies,  and  private-sector  health 

providers  will  use  the  DMIX  infrastructure  and  services  to: 

•  Share  standardized  health  data  using  standard  terminology 

•  Securely  and  reliably  exchange  standardized  electronic  health 
data  with  all  partners 

•  Access  a  patient’s  medical  history  from  a  single  platform, 
eliminating  the  need  to  access  separate  systems  to  obtain 
patient  information 

•  Maintain  continuity  of  care 

•  Exchange  outpatient  pharmacy  and  medication  allergy  data 
and  check  for  drug-to-drug  and  drug-to-allergy  interaction 

Major  Contractors 

•  Data  Federation/JLV:  Hawaii  Resource  Group  -  Honolulu, 
Hawaii 

•  Test  Support:  Deloitte  -  Falls  Church,  Virginia 

•  Program  Manager  support:  Technatomy  -  Fairfax,  Virginia 


Activity 

•  During  FY15,  the  USAMEDDBD  and  ATEC  conducted 
operational  tests  of  DMIX  Release  2  and  iEHR  Increment  1 
in  separate  events.  The  USAMEDDBD  conducted  an 
operational  assessment  of  DMIX  Release  2  because 

the  DMIX  program  manager  will  implement  VLER, 
a  major  capability  of  the  system,  in  DMIX  Release  3. 
USAMEDDBD  plans  to  conduct  an  MOT&E  of  DMIX 
Release  3  with  the  VLER  capability  and  operational 
commercial  health  care  partners. 

DMIX  Release  2 

•  From  April  20,  2015,  through  May  1,  2015,  the 
USAMEDDBD  and  ATEC  conducted  an  operational 
assessment  of  DMIX  Release  2  in  accordance  with  a 
DOT&E-approved  test  plan  to  support  a  Milestone  C 
decision  for  the  JLV  Viewer  component  of  the  DMIX 
program. 

•  SPAWAR  conducted  a  cybersecurity  CVPA  of  the  system 
in  the  developmental  test  environment.  The  assessment 
concluded  in  June  2015. 

•  The  PMO  coordinated  with  USAMEDDBD  and  the  San 
Antonio  Military  Medical  Center  to  schedule  a  VCD  of 
DMIX  in  December  2015  to  verify  corrective  actions  for 
Release  2  high-severity  defects. 


DMIX  Release  3 

•  After  success  III!  developmental  testing,  the  Program 
Executive  Office  DHMS  deployed  DMIX  Release  3  on 
September  18,  2015,  to  implement  fixes  to  system  defects 
discovered  during  DMIX  Release  2  operational  testing  and 
to  prepare  for  the  Release  3  MOT&E. 

•  Following  deployment,  users  discovered  two  problems 
associated  with  theater  inpatient  data  and  progress  notes 
not  displaying  properly  in  JLV.  Users  also  identified  an 
Essentris  site  that  had  made  local  changes,  which  prevented 
the  site’s  users  from  accessing  patient  data  from  DMIX. 

The  DMIX  program  manager  stated  that  she  has  developed 
a  plan  to  resolve  the  DMIX  Release  3  system  defects  and 
data  accessibility  problem. 

iEHR  Increment  1 

•  SPAWAR  conducted  a  cybersecurity  CVPA  of  iEHR 
Increment  1  from  October  27,  2014,  through  December  10, 

2014,  and  an  Adversarial  Assessment  from  January  12  -  16, 

2015. 

Terminology  Mapping 

•  Both  the  DOD  and  VA  had  planned  to  provide  health 
data  mapped  to  a  medical  terminology  standard  through 
the  JLV  to  help  users  read  and  analyze  patient  data  more 
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easily.  There  are  20  Clinical  Domains,  which  have  national 
standards  for  terms  within  the  domain.  In  December  2013, 
the  DOD  and  VA  delivered  terminology  maps  for  seven 
Clinical  Domains  to  support  DMIX  Release  0.  In  2014, 
the  DMIX  program  manager  implemented  terminology 
maps  for  14  Clinical  Domains;  in  FY15,  completed  delivery 
of  the  remaining  Clinical  Domain  maps  and  provided 
maintenance  of  active  terms  for  another  11  Clinical 
Domains.  The  VA  implemented  terminology  maps  for  one 
additional  domain  in  December  2014. 

•  DOT&E  is  working  with  the  DMIX  PMO,  the  Interagency 
Program  Office,  and  the  operational  test  community  to 
develop  an  operational  test  methodology  to  determine  the 
accuracy  and  completeness  of  Clinical  Domain  terminology 
maps. 

Assessment 

DMIX  Release  2 

•  DMIX  Release  2  was  not  operationally  effective  because 
of  high-severity  defects  that  included  (1)  inconsistent 
query  results  for  which  the  system  did  not  always  display 
all  records  available  in  a  date  range,  (2)  medical  record 
notes  not  displaying  due  to  the  format  of  the  original  note, 
and  (3)  inconsistencies  in  data  displays  and  navigation 
buttons  between  different  schemes  within  the  DMIX 
viewer.  DOT&E  sent  a  memorandum  to  the  Program 
Executive  Office  DHMS  on  May  8,  2015,  which  detailed 
the  Severity  1  defect  causing  inconsistent  query  results 

in  a  date  range.  Although  the  high-severity  defects 
affected  data  completeness,  all  data  displayed  in  JLV  were 
accurate  and  user  queries  of  patient  records  were  timely. 

The  Program  Office  took  immediate  steps  to  correct  the 
problems  and  operational  testers  plan  to  verify  the  fixes  in 
December  2015.  The  VCD  results  will  be  included  in  the 
2016  Annual  Report. 

•  The  PMO  improved  DMIX  developmental  testing  to 
evaluate  the  system’s  response  to  invalid  inputs.  DOT&E 
recommended  such  testing  to  discover  failure  modes  caused 
by  non-typical  use  cases,  such  as  those  observed  in  DMIX 
Release  2  operational  testing,  earlier  in  the  test  cycle. 


•  DMIX  Release  2  is  operationally  suitable.  Users  liked  the 
functionality  and  indicated  that  it  improved  their  business 
processes.  Additionally,  users  rated  the  system  highly  for 
usability  and  liked  the  data  display,  record  selection  and 
customization  capabilities,  and  supporting  documentation. 

•  The  cybersecurity  assessment  identified  significant 
vulnerabilities  that  could  result  in  the  loss  of  confidentiality, 
integrity,  or  availability  of  personal  health  information  and 
personally  identifiable  information. 

DMIX  Release  3 

•  The  DOD  and  VA  deployed  DMIX  Release  3  in  September 
2015  to  all  JLV  users.  The  user  base  has  significantly 
expanded  from  3,000  users  earlier  this  year  to  an  estimated 
17,000  users.  Program  risk,  and  potential  risk  to  patient 
safety,  increased  following  the  deployment  of  Release  3 
when  users  discovered  problems  with  theater  inpatient  data, 
progress  notes  not  displaying  properly,  and  with  local  data 
access. 

•  The  DMIX  program  manager  has  scheduled  an  MOT&E 
of  DMIX  Release  3  in  March  2016,  creating  a  gap  of  six 
months  between  deployment  and  operational  testing.  The 
long  gap  between  deployment  and  operational  testing 
further  increases  the  risk  of  DMIX  users  experiencing 
system  problems  prior  to  the  MOT&E. 

iEHR  Increment  1 

•  The  cybersecurity  assessment  of  iEHR  Increment  1 
identified  vulnerabilities  that  could  compromise  medical 
information  if  not  quickly  and  adequately  addressed. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  DOD  and  DMIX 
PMO  have  addressed  the  FY14  recommendations;  however, 
the  VA  was  not  able  to  accelerate  clinical  terminology  mapping 
efforts.  The  VA  implemented  terminology  maps  for  one 
additional  domain  in  December  2014. 

•  FY15  Recommendation. 

1 .  The  DMIX  PMO  should  conduct  further  cybersecurity 
testing  on  the  operational  DMIX  system  to  verify  fixes  and 
mitigations  of  the  vulnerabilities  found  during  testing  of 
DMIX  Release  2. 


DMIX 
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Defense  Readiness  Reporting  System  -  Strategic 

(DRRS-S) 


DRRS-N 


DRRS-A 


Executive  Summary 

•  The  Joint  Interoperability  Test  Command 
(JITC)  conducted  the  Defense  Readiness 
Reporting  System  -  Strategic  (DRRS-S) 
IOT&E  from  May  2015  through  June  2015. 
Emerging  residts  identified  significant  system 
and  end-to-end  process  deficiencies.  The 
DRRS-S  program  manager  requested  an 
extension  of  the  IOT&E  through  October  2015 
to  correct  system  deficiencies  and  allow  JITC 
to  independently  validate  the  fixes.  DOT&E 
agreed  to  the  extension. 

•  JITC  continued  IOT&E  in  September  and 
October  2015.  DOT&E  plans  to  submit  a 
DRRS-S  IOT&E  report  in  2QFY16. 


System 

•  DRRS-S  is  a  Secret  Internet  Protocol  Router 
Network-accessible  web  application  designed 
to  replace  the  Global  Status  of  Resources 
and  Training  System,  a  Force  Readiness 
component  of  Global  Command  and  Control 
System  -  Joint. 

•  DRRS-S  production  and  backup  systems 
are  hosted  at  separate  Defense  Enterprise 
Computing  Centers  on  commercial 
off-the-shelf  hardware  consisting  of  application  and  database 
server  enclaves  using  Microsoft  Windows®  operating  systems. 

•  DRRS-S  receives  and  processes  readiness  reports  and 
data  from  Service-specific  increments  of  the  larger  DRRS 
enterprise,  including  DRRS-Army,  DRRS-Marine  Corps,  and 
DRRS-Navy.  Combatant  Commanders,  and  the  subordinates 
they  direct,  DOD  agencies,  and  Air  Force  units  report  directly 
within  DRRS-S. 


Service  Authoritative  Data  Sources 
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AFRICOM  -  Africa  Command 
CCMDs  -  Combatant  Commanders 
CENTCOM  -  Central  Command 

DRRS-A  -  Defense  Readiness  Reporting  System  -  Army 
DRRS-MC  -  Defense  Readiness  Reporting  System  -  Marine  Corps 
DRRS-N  -  Defense  Readiness  Reporting  System  -  Navy 
EUCOM  -  European  Command 

GCCS-J  -  Global  Command  and  Control  System  -  Joint 
GFM-DI  -  Global  Force  Management  Data  Initiative 
GFMTS  -  Global  Force  Management  Toolset 
JDNETS  -  JOPES  Data  Network  Services 
JSUB  -  JOPES  Subscriptions 

JOPES  -  Joint  Operational  Planning  and  Execution  System 
JTIMS  -  Joint  Training  Information  Management  System 


Force  Structure 
Data  (Future) 


JUM  -  Joint  User  Messaging 

MET  -  Mission  Essential  Task 

METL  -  Mission  Essential  Task  List 

NGB  -  National  Guard  Bureau 

NORTHCOM  -  Northern  Command 

PACOM  -  Pacific  Command 

SECDEF  -  Secretary  of  Defense 

SIPRNET  -  Secret  Internet  Protocol  Router  Network 

SOCOM  -  Special  Operations  Command 

SORTS  -  Status  of  Resources  and  Training  System 

SOUTHCOM  -  Southern  Command 

STRATCOM  -  Strategic  Command 

TRANSCOM  -  Transportation  Command 


DRRS  collaborative  environment  to  evaluate  the  readiness 
and  capability  of  U.S.  Armed  Forces  to  carry  out  assigned  and 
potential  tasks. 

Reporting  organizations  input  both  mission  readiness  and 
unit  readiness  data,  such  as  Global  Status  of  Resources  and 
Training  System  data,  into  DRRS-S  and  use  it  to  make  mission 
readiness  assessments  against  standardized  missions  and  tasks. 


Mission 

•  Combatant  Commanders,  Military  Services,  Joint  Chiefs  of 
Staff,  Combat  Support  Agencies,  and  other  key  DOD  users 
(such  as  the  Secretary  of  Defense  and  National  Guard)  use  the 


Major  Contractor 

InnovaSystems  International,  LLC  -  San  Diego,  California 


Activity 

•  From  May  2015  through  June  2015,  JITC  conducted  an 
IOT&E  in  accordance  with  the  DOT&E-approved  test  plan. 
IOT&E  revealed  a  number  of  significant  deficiencies  with 
the  system  and  end-to-end  data  management  processes. 
Therefore,  the  DRRS-S  program  manager  requested  an 


extension  of  the  IOT&E  through  October  2015  to  allow  for 
correction  of  system  deficiencies  and  provide  sufficient  time 
for  JITC  to  independently  verify  the  fixes.  DOT&E  agreed  to 
the  extension. 
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•  JITC  continued  IOT&E  in  September  and  October  2015.  This 
test  window  included  two  monthly  readiness  reporting  cycles 
to  verify  the  accuracy,  completeness,  and  timeliness  of  Service 
readiness  reports. 

•  JITC  and  the  Army  Research  Laboratory  Survivability  and 
Lethality  Assurance  Directorate  conducted  a  Cooperative 
Vulnerability  and  Penetration  cybersecurity  assessment  from 
February  2015  through  May  2015.  The  Defense  Information 
Systems  Agency  Risk  Management  Executive  Red  Team 
conducted  a  cybersecurity  Adversarial  Assessment  in 

June  2015. 

•  DOT&E  will  submit  an  IOT&E  report  on  the  DRRS-S  in 
2QFY16. 


Assessment 

The  DRRS-S  Data  Authentication  Group  reviewed  and 
authenticated  IOT&E  data  in  November  2015.  DOT&E  began 
evaluating  the  IOT&E  test  data  in  November  2015.  DOT&E 
plans  to  submit  a  DRRS-S  IOT&E  report  in  2QFY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  DRRS-S  Program 
Office  addressed  all  previous  recommendations. 

•  FY15  Recommendations.  None. 
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Department  of  Defense  (DOD)  Teleport 


Executive  Summary 

•  The  Defense  Information  Systems  Agency  (DISA)  is 
developing  the  Teleport  Generation  3  Phase  3  (G3P3) 
capability  that  is  intended  to  provide  interconnectivity 
between  legacy  Ultra  High  Frequency  (UHF)  radios,  which 
provide  half  duplex,  push-to-talk  service  and  the  Mobile  User 
Objective  System  (MUOS)  radios,  which  can  provide  full 
duplex  service.  To  achieve  the  G3P3  capability,  the  program 
manager  is  adding  two  new  components  to  the  Teleport 
architecture,  the  MUOS  to  Legacy  Gateway  Component 
(MLGC)  and  MUOS  Voice  Gateway  (MVG).  The  program 
manager  is  planning  to  install  the  MLGC  at  five  of  the  six 
primary  Teleport  sites  and  the  MVG  at  the  Virginia  and 
Hawaii  Teleport  sites,  collocated  with  two  MUOS  Radio 
Access  Facilities. 

•  The  Joint  Interoperability  Test  Command  (JITC)  conducted 
the  G3P3  operational  assessment  (OA)  from  June  22,  2015, 
to  August  11,  2015,  at  the  Northwest  Virginia  Teleport  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan  and  test  plan.  The  OA  is  intended  to  inform  a 
Milestone  C  decision  by  the  DISA  Component  Acquisition 
Executive  planned  for  February  2016. 

•  The  MLGC  and  MVG  operated  as  intended  in  the  limited 
operational  environment  afforded  during  the  OA  period. 
Government  program  personnel  and  contractors  completed 
data  exchanges  through  the  MLGC  with  completion  rates 
of  98  percent  relative  to  an  88  percent  threshold  criterion. 
Government  program  personnel  and  contractors  completed 
voice  exchanges  through  MLGC  and  MVG  at  completion 
rates  of  94  percent  relative  to  the  88  percent  threshold 
criteria.  Additional  developmental  testing  is  required  prior  to 
operational  testing  in  1QFY17. 

System 

•  DOD  Teleport  sites  are  globally-distributed  Satellite 
Communication  (SATCOM)  facilities.  There  are  six  core 
Teleport  facilities  located  in  Virginia,  Germany,  Italy,  Japan, 
Hawaii,  and  California,  and  three  secondary  facilities  located 
in  Bahrain,  Australia  (future),  and  Guam.  Teleport  sites 
consist  of  four  segments: 

-  The  radio  frequency  segment  consists  of  SATCOM 
earth  terminals  that  operate  in  UHF,  X,  C,  Ku,  Ka, 
and  Extremely-High  Frequency  bands.  The  terminals 
provide  radio  frequency  links  between  the  Teleport  site 
and  the  deployed  user  SATCOM  terminal  via  military  or 
commercial  satellites. 


MUOS  EARTH  TERMINALS 
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-  The  baseband  segment  includes  encryption,  switching, 
multiplexing,  and  routing  functions  for  connecting  data 
streams  or  packetized  data  to  the  terrestrial  Defense 
Information  Systems  Network  (DISN). 

-  The  network  services  segment  provides  connectivity  to 
the  DISN  long-haul  networks  and  other  internet-working 
functions  necessary  to  meet  the  user’s  requirements. 

-  The  management  control  segment  provides  centralized 
monitoring  and  control  of  Teleport  baseband  hardware, 
earth  terminal  hardware,  transmission  security,  and  test 
equipment. 

•  Teleport  provides  deployed  forces  access  to  standard  fixed 

gateways  from  anywhere  in  the  world  for  all  six  DISN 

services: 

-  Secret  Internet  Protocol  Router  Network 

-  Non-classified  Internet  Protocol  Router  Network 

-  Defense  Red  Switch  Network 

-  Defense  Switched  Network 

-  Video  Teleconference 

-  Joint  Worldwide  Intelligence  Communications  System 

Mission 

Combatant  Commanders,  Services,  and  deployed  operational 
forces  use  DOD  Teleport  systems  in  all  phases  of  conflict  to  gain 
access  to  worldwide  military  and  commercial  SATCOM  services. 

Major  Contractor 

Government  Integrator:  Defense  Information  Systems  Agency 
(DISA) 
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Activity 

•  DISA  is  developing  the  Teleport  G3P3  capability  that  is 
intended  to  provide  interconnectivity  between  legacy  UHF 
radios,  which  provide  half  duplex,  push-to-talk  service  and 
MUOS  radios,  which  have  the  capability  to  provide  full 
duplex  service.  To  achieve  the  G3P3  capability,  the  program 
manager  is  adding  two  new  components  to  the  Teleport 
architecture,  the  MLGC  and  MVG.  The  program  manager 
is  planning  to  install  the  MLGC  at  five  of  the  six  primary 
Teleport  sites  and  the  MVG  at  the  Virginia  and  Hawaii 
Teleport  sites,  collocated  with  two  MUOS  Radio  Access 
Facilities. 

•  In  March  2015,  DISA  installed  a  MLGC  test  suite  at  the 
DOD  Teleport  Test  Lab  within  the  Army’s  Joint  Satellite 
Engineering  Center,  at  Aberdeen  Proving  Ground,  Maryland, 
and  at  the  Northwest  Virginia  Teleport  site  for  integrated 
testing. 

•  JITC  conducted  the  G3P3  OA  from  June  22,  2015,  through 
August  11,  2015,  at  the  Northwest  Virginia  Teleport  in 
accordance  with  the  DOT&E-approved  Test  and  Evaluation 
Master  Plan  and  test  plan.  The  OA  informs  a  Milestone  C 
decision  by  the  DISA  Component  Acquisition  Executive 
planned  for  February  2016. 

-  JITC  testers  observed  set-up,  management,  and  tear-down 
of  services  at  the  DOD  Northwest  Virginia  Teleport, 

and  communication  between  deployed  users  at  Saint 
Julien’s  Creek,  Virginia,  and  the  MUOS  Radio  Integration 
Laboratory  in  San  Diego,  California.  Operators  using 
two  legacy  UHF  radios  and  two  MUOS-capable  terminals 
located  at  Saint  Julien’s  Creek  exchanged  data  and  voice 
with  two  MUOS-capable  terminals  located  at  the  MUOS 
Radio  Integration  Laboratory.  Contractor  and  government 
subject  matter  experts  configured  the  Teleport  equipment 
and  operated  the  deployed  radios. 

-  Operational  testers  collected  equipment  uptime  and 
downtime  to  support  an  assessment  of  reliability  and 
availability. 

-  In  conjunction  with  the  OA,  JITC  conducted  a  cooperative 
cybersecurity  assessment  from  September  28,  2015, 
through  October  2,  2015,  at  the  Northwest  Virginia 
Teleport  site. 

•  DOT&E  submitted  the  DOD  Teleport  System,  G3P3 
Operational  Assessment  in  December  2015. 


•  JITC  plans  to  conduct  an  operational  test  of  the  G3P3 
capability  in  1QFY17. 

Assessment 

•  The  OA  provided  a  limited  operational  environment.  The 
MLGC  and  MVG  test  suites  installed  at  the  Northwest 
Teleport  were  configured  and  operated  by  contractor 
personnel.  Teleport  operators  were  not  contractually  allowed 
to  configure  the  MLGC  or  MVG  prior  to  government 
acceptance  of  the  equipment. 

•  The  MLGC  and  MVG  operated  as  intended  in  the  limited 
operational  environment  afforded  during  this  test  period. 
Government  program  personnel  and  contractors  completed 
data  exchanges  through  the  MLGC  with  completion  rates 
of  98  percent  relative  to  an  88  percent  threshold  criterion. 
Government  program  personnel  and  contractors  completed 
voice  exchanges  through  MLGC  and  MVG  at  completion  rates 
of  94  percent  relative  to  the  88  percent  threshold  criteria. 

•  The  JITC  cooperative  cybersecurity  assessment  identified 
potential  vulnerabilities  that  could  degrade  system  security. 

•  Although  the  MLGC  and  MVG  performed  well,  the  limited 
data  from  this  test  precludes  the  MLGC  and  MVG  meeting 
reliability  and  availability  requirements  with  confidence. 

Based  upon  the  scope  of  the  OA  testers  were  not  able  to 
evaluate  maintainability  and  documentation. 

•  The  Teleport  Test  and  Evaluation  Master  Plan  needs  to  be 
updated  to  support  the  G3P3  Milestone  C  decision  planned  for 
February  2016  and  support  the  G3P3  operational  test  planned 
for  1QFY17. 

Recommendations 

•  Status  of  Previous  Recommendations.  DISA  has  satisfactorily 
addressed  all  previous  recommendations. 

•  FY15  Recommendation. 

1 .  DISA  should  update  the  Teleport  Test  and  Evaluation 
Master  Plan  before  the  G3P3  Milestone  C  decision  planned 
for  2QFY16  and  the  G3P3  operational  test  planned  for 
1QFY17. 
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F-35  Joint  Strike  Fighter  (JSF) 


Executive  Summary 

Test  Planning,  Activity,  and  Assessment 

•  The  program  focused  on  culminating  Block  2B  development 
and  testing  in  order  to  provide  a  fleet  release  enabling  the 
Marine  Corps  F-35B  Joint  Strike  Fighter  (JSF)  declaration 
of  Initial  Operational  Capability  (IOC),  while  transitioning 
development  and  flight  test  resources  to  Block  3i  and 
Block  3F. 

-  The  program  terminated  Block  2B  developmental  flight 
testing  in  May  2015,  delivering  Block  2B  capability 
with  deficiencies  and  limited  combat  capability.  The 
Marine  Corps  declared  IOC  at  the  end  of  July  2015. 
However,  if  used  in  combat,  the  Block  2B  F-35  will  need 
support  from  command  and  control  elements  to  avoid 
threats,  assist  in  target  acquisition,  and  control  weapons 
employment  for  the  limited  weapons  carriage  available 
(i.e.,  two  bombs,  two  air-to-air  missiles).  Block  2B 
deficiencies  in  fusion,  electronic  warfare,  and  weapons 
employment  result  in  ambiguous  threat  displays,  limited 
ability  to  respond  to  threats,  and  a  requirement  for 
off-board  sources  to  provide  accurate  coordinates  for 
precision  attack.  Since  Block  2B  F-35  aircraft  are  limited 
to  two  air-to-air  missiles,  they  will  require  other  support 
if  operations  are  contested  by  enemy  fighter  aircraft.  The 
program  deferred  deficiencies  and  weapons  delivery 
accuracy  (WDA)  test  events  from  Block  2B  to  Block  3i 
and  Block  3F,  a  necessary  move  in  order  to  transition  the 
testing  enterprise  to  support  Block  3i  flight  testing  and 
Block  3F  development,  both  of  which  began  later  than 
planned  in  the  program’s  Integrated  Master  Schedule 
(IMS). 

-  Block  3i  developmental  flight  testing  restarted  for  the 
third  time  in  March  2015,  after  two  earlier  starts  in 
May  and  September  2014.  Block  3i  developmental 
flight  testing  completed  in  October,  eight  months  later 
than  planned  by  the  program  after  restructuring  in  2012, 
as  reflected  in  the  IMS.  Block  3i  began  with  re-hosting 
immature  Block  2B  software  and  capabilities  into  avionics 
components  with  new  processors.  Though  the  program 
originally  intended  that  Block  3i  would  not  introduce  new 
capabilities  and  not  inherit  technical  problems  from  earlier 
blocks,  this  is  what  occurred.  The  Air  Force  insisted  on 
fixes  for  five  of  the  most  severe  deficiencies  inherited 
from  Block  2B  as  a  prerequisite  to  use  the  final  Block 

3i  capability  in  the  Air  Force  IOC  aircraft;  Air  Force 
IOC  is  currently  planned  for  August  2016  (objective)  or 
December  2016  (threshold).  However,  Block  3i  struggled 
during  developmental  testing  (DT),  due  to  the  inherited 
deficiencies  and  new  avionics  stability  problems.  Based 
on  these  Block  3i  performance  issues,  the  Air  Force 
briefed  that  Block  3i  mission  capability  is  at  risk  of  not 


meeting  IOC  criteria  to  the  Joint  Requirements  Oversight 
Council  (JROC)  in  December  2015.  The  Air  Force 
recently  received  its  first  Block  3i  operational  aircraft  and 
is  assessing  the  extent  to  which  Block  3i  will  meet  Air 
Force  IOC  requirements;  this  assessment  will  continue  into 
mid-2016. 

-  Block  3F  developmental  flight  testing  began  in 
March  2015,  11  months  later  than  the  date  planned  by 
the  program  after  restructuring  in  2012,  as  reflected  in 
the  IMS.  Progress  has  been  limited  (flight  testing  has 
accomplished  approximately  12  percent  of  the  Block  3F 
baseline  test  points  as  of  the  end  of  November)  as  the 
program  focused  on  closing  out  Block  3i  testing  and 
providing  a  software  version  suitable  to  support  plans  for 
the  Air  Force  to  declare  IOC  in  August  2016. 

•  The  current  schedule  to  complete  System  Development  and 
Demonstration  (SDD)  and  enter  IOT&E  by  August  2017  is 
unrealistic. 

-  Full  Block  3F  mission  systems  development  and  testing 
cannot  be  completed  by  May  20 1 7,  the  date  reflected 

in  the  most  recent  Program  Office  schedule,  which  is 
seven  months  later  than  the  date  planned  after  the  2012 
restructure  of  the  program.  Although  the  program  has 
recently  acknowledged  some  schedule  pressure  and  began 
referencing  July  31,  2017,  as  the  end  of  SDD  flight  test, 
that  date  is  unrealistic  as  well.  Instead,  the  program  will 
likely  not  finish  Block  3F  development  and  flight  testing 
prior  to  January  2018,  an  estimate  based  on  the  following 
assumptions: 

■  Continuing  a  six  test  point  per  flight  accomplishment 
rate,  which  is  equal  to  the  calendar  year  2015  (CY15) 
rate  observed  through  the  end  of  November. 
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■  Continuing  a  flight  rate  of  6.8  flights  per  month,  as  was 
achieved  through  the  end  of  November  2015,  exceeding 
the  planned  rate  of  6  flights  per  month  (note  that  if  the 
flight  rate  deteriorates  to  the  planned  rate  of  6  flights  per 
month,  then  testing  will  not  complete  until  May  2018). 

■  Completing  the  full  Block  3F  test  plan  (i.e.,  all  7,230 
original  baseline  and  budgeted  non-baseline  test  points 
in  the  Block  3F  joint  test  plan). 

■  Continuing  the  CY15  discovery  rate  of  5  percent, 
i.e.,  5  additional  test  points  are  required  to  address 
new  discoveries  per  100  baseline  test  points 
accomplished.  This  assumption  is  optimistic.  In  the 
likely  event  significant  new  discoveries  continue  during 
developmental  testing  in  2016,  additional  Block  3F 
software  releases  would  be  needed  to  address  them, 
adding  more  test  points  and  extending  development 
further. 

-  The  program  could,  as  has  been  the  case  in  testing 
previous  software  increments,  determine  that  test  points 
in  the  plan  are  no  longer  required  for  the  Block  3F  fleet 
release.  However,  the  program  will  need  to  ensure  that 
deleting  and/or  deferring  Block  3F  testing  before  the  end 
of  SDD  and  start  of  IOT&E  does  not  result  in  increasing 
the  likelihood  of  discovery  of  deficiencies  in  IOT&E  or 
degrading  F-35  combat  capability.  Whatever  capability 
the  program  determines  as  ready  for  IOT&E  will  undergo 
testing  fully  consistent  with  the  Department’s  threat 
assessments,  war  plans,  and  the  Services’  concepts  of 
operation. 

The  program  has  proposed  a  “block  buy”  that  commits 
to  and  combines  procurement  of  three  lots  of  aircraft  to 
gain  savings.  Executing  the  “block  buy”  would  require 
commitments  to  procuring  as  many  as  270  U.S.  aircraft, 
as  well  as  commitments  by  foreign  partners  to  purchasing 
substantial  numbers  of  aircraft.  Depending  upon  the  timing, 
it  is  possible  a  commitment  to  the  “block  buy”  would  be 
made  before  operational  testing  is  complete.  In  that  case, 
entering  a  “block  buy”  would  raise  the  following  questions: 

-  Is  it  premature  to  commit  to  the  “block  buy”  given  that 
significant  discoveries  requiring  correction  before  F-35’s 
are  used  in  combat  are  occurring,  and  will  continue 

to  occur,  throughout  the  remaining  developmental 
and  operational  testing?  The  program  continues  to 
struggle  with  Block  3F  developmental  testing,  and  in 
December  2015  the  Air  Force  rated  its  proposed  initial 
operational  capability  supported  by  Block  3i  as  “red”  due 
to  the  problems  ongoing  testing  has  revealed. 

-  Is  it  prudent  to  further  increase  substantially  the  number 
of  aircraft  bought  that  may  need  modifications  to  reach 
full  combat  capability  and  service  life?  As  the  program 
manager  has  noted,  essentially  every  aircraft  bought  to 
date  requires  modifications  prior  to  use  in  combat. 

-  Would  committing  to  a  “block  buy”  prior  to  the 
completion  of  IOT&E  provide  the  needed  incentives  to  the 
contractor  and  the  Program  Office  to  correct  an  already 
substantial  list  of  deficiencies  in  performance,  a  list  that 


will  only  lengthen  as  Block  3F  testing  continues  and 
IOT&E  is  conducted? 

-  Would  entering  a  “block  buy”  contract  prior  to  the 
completion  of  IOT&E  be  consistent  with  the  “fly  before 
you  buy”  approach  to  defense  acquisition  that  many  in  the 
Administration  have  supported?  Similarly,  would  such  a 
“block  buy”  be  consistent  with  the  intent  of  Title  10  U.S. 
Code,  which  stipulates  that  IOT&E  must  be  completed 
and  a  report  on  its  results  provided  to  Congress  before 
committing  to  Full-Rate  Production — a  commitment  that 
some  could  argue  would  be  made  by  executing  the  “block 
buy?” 

Helmet  Mounted  Display  System  (HMDS) 

•  The  program  tested  the  Generation  III  (Gen  III) 
helmet-mounted  display  system  (HMDS),  which  is  intended 
to  resolve  all  of  the  deficiencies  discovered  in  the  Gen  II 
system  in  prior  years.  The  Gen  III  system  is  a  requirement 
for  Air  Force  IOC  in  2016;  it  will  be  the  helmet  used  to 
complete  SDD  and  IOT&E.  After  Gen  III  developmental 
testing,  developmental  test  pilots  reported  less  jitter,  proper 
alignment,  improved  ability  to  set  symbology  intensity,  less 
latency  in  imagery  projections,  and  improved  performance 
of  the  night  vision  camera.  However,  operational  testing  in 
realistic  conditions  and  mission  task  levels,  including  gun 
employment,  is  required  to  determine  if  further  adjustments 
are  needed. 

Mission  Data  Load  Development  and  Testing 

•  The  F-35  relies  on  mission  data  loads — which  are  a 
compilation  of  the  mission  data  files  needed  for  operation 
of  the  sensors  and  other  mission  systems — to  work  in 
conjunction  with  the  system  software  data  load  to  drive 
sensor  search  parameters  and  to  identify  and  correlate  sensor 
detections,  such  as  threat  and  friendly  radar  signals.  The 
U.S.  Reprogramming  Lab  (USRL),  a  U.S.  government 

lab,  produces  these  loads  for  U.S.  operational  and  training 
aircraft.  Mission  data  optimization  testing,  which  includes 
both  lab-testing  and  flight-testing,  is  conducted  by  an  Air 
Force  operational  test  unit  augmented  by  Navy  personnel. 
The  unit  provides  the  test  plans  to  the  DOT&E  for  approval 
and  independent  oversight. 

•  Significant  deficiencies  exist  in  the  USRL  that  preclude 
efficient  development  and  adequate  testing  of  effective 
mission  data  loads  for  Block  3F.  Despite  being  provided 
a  $45  Million  budget  in  FY13,  the  program  has  still 

not  designed,  contracted  for,  and  ordered  the  required 
equipment — a  process  that  will  take  at  least  two  years,  not 
counting  installation  and  check-out.  In  addition,  despite  the 
conclusions  of  a  study  by  the  Program  Office  indicating  that 
substantial  upgrades  are  needed  to  the  laboratory’s  hardware, 
the  program  is  currently  only  pursuing  a  significantly  lesser 
upgrade  due  to  budgetary  constraints.  This  approach  would 
leave  the  USRL  with  less  capability  than  the  F-35  Foreign 
Military  Sales  Reprogramming  Lab.  Unless  remedied, 
these  deficiencies  in  the  USRL  will  translate  into  significant 
limitations  for  the  F-35  in  combat  against  existing  threats. 
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The  program  must  take  immediate  action  to  complete 
required  modifications  and  upgrades  to  the  lab  before  the 
USRL  is  required  to  provide  the  Block  3F  mission  data  load 
for  tactics  development  and  preparations  for  IOT&E. 

•  After  the  program  delayed  the  build-up  of  the  USRL 
equipment  and  software  tools,  which  created  schedule 
pressure  on  Block  2B  mission  data  load  development  and 
testing,  the  Program  Office  forced  the  USRL  to  truncate 
the  planned  testing,  forgoing  important  steps  in  mission 
data  load  development  in  order  to  provide  a  limited 
mission  data  load  in  June  2015  for  the  Marine  Corps  IOC 
declaration  in  July  2015.  Fielded  operational  units  must 
take  into  consideration  the  limited  extent  of  lab  and  flight 
testing  that  occurred — which  creates  uncertainties  in 
F-35  effectiveness — until  the  USRL  is  able  to  complete 
development  and  testing  of  a  Block  2B  mission  data  load. 
This  is  planned  to  occur  in  early  2016. 

Weapons  Integration 

•  The  program  terminated  Block  2B  developmental  testing 
for  weapons  integration  in  December  2015  after  completing 
12  of  the  15  planned  WDA  events.  The  program  planned  to 
complete  all  15  WDA  events  by  the  end  of  October  2014, 
but  delays  in  implementing  software  fixes  for  deficient 
performance  of  mission  systems  sensors  and  fusion  delayed 
progress.  Three  events  were  deferred  to  Block  3i  (one  event) 
and  Block  3F  (two  events)  developmental  testing. 

-  Eleven  of  the  12  events  required  intervention  by  the 
developmental  test  control  team  to  overcome  system 
deficiencies  and  ensure  a  successful  event  (i.e.,  acquire 
and  identify  the  target  and  engage  it  with  a  weapon). 

The  program  altered  the  event  scenario  for  three  of 
these  events,  as  well  as  the  twelfth  event,  specifically  to 
work  around  F-35  system  deficiencies  (e.g.,  changing 
target  spacing  or  restricting  target  maneuvers  and 
countermeasures). 

-  The  performance  of  the  Block  2B-configured  F-35,  if  used 
in  combat,  will  depend  in  part  on  the  degree  to  which  the 
enemy’s  capabilities  exceed  the  constraints  of  these  narrow 
scenarios  and  the  operational  utility  of  the  workarounds 
necessary  for  successful  weapons  employment. 

•  The  Block  3F  WDA  events  plan  currently  contains  events 
that  will  test  Block  3F  capabilities  to  employ  the  GBU-12 
Paveway  II  laser-guided  bomb,  GBU-31/32  Joint  Direct 
Attack  Munition  (JDAM),  Navy  Joint  Stand-off  Weapon 
(JSOW)-Cl,  Small  Diameter  Bomb  I  (SDB-1),  AIM-120C 
Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM), 
AIM-9X,  and  the  gun  in  the  full  operating  environment  of 
each  variant. 

-  The  Block  3F  developmental  test  WDA  plan  contains 
48  events  in  the  approved  Test  and  Evaluation  Master 
Plan  (TEMP),  plus  two  WDA  events  deferred  from 
Block  2B,  for  a  total  of  50.  These  50  WDA  events  cannot 
be  accomplished  within  the  remaining  time  planned  by 
the  Program  Office  to  complete  Block  3F  flight  test  (by 
May  2017,  per  the  program’s  master  schedule),  nor  by  July 
2017  (the  most  recent  briefed  date  to  complete  Block  3F 


flight  test  from  the  Program  Office),  and  support  the  date 
in  the  IMS  for  the  Block  3F  fleet  release  (August  2017). 
The  past  WDA  event  execution  rate  is  approximately  one 
event  per  month.  The  test  team  would  need  to  triple  this 
rate  to  complete  all  WDA  events  in  the  approved  TEMP 
by  May  2017.  However,  these  Block  3F  events  are  more 
complex  than  the  Block  2B  and  3i  events. 

-  In  an  attempt  to  meet  the  schedule  requirements  for 
weapon  certification,  the  Program  Office  has  identified 
10  WDA  events  for  the  F-35A  and  5  events  for  the 
F-35B  and  F-35C  that  must  be  accomplished  during 
Block  3F  developmental  testing.  The  program  still 
plans  to  accomplish  the  remaining  33  events,  if  schedule 
margin  allows.  The  overall  result  of  the  WDA  events 
must  be  that  the  testing  yields  sufficient  data  to  evaluate 
Block  3F  capabilities.  Deleting  numerous  WDA  events 
puts  readiness  for  operational  testing  and  employment  in 
combat  at  significant  risk. 

Verification  Simulation  (VSim) 

•  Due  to  inadequate  leadership  and  management  on  the  part 
of  both  the  Program  Office  and  the  contractor,  the  program 
has  failed  to  develop  and  deliver  a  Verification  Simulation 
(VSim)  for  use  by  either  the  developmental  test  team  or  the 
JSF  Operational  Test  Team  (JOTT),  as  has  been  planned  for 
the  past  eight  years  and  is  required  in  the  approved  TEMP. 
Neither  the  Program  Office  nor  the  contractor  has  accorded 
priority  to  VSim  development  despite  early  identification  of 
requirements  by  the  JOTT,  $250  Million  in  funding  added 
after  the  Nunn-McCurdy-driven  restructure  of  the  program 
in  2010,  warnings  that  development  and  validation  planning 
were  not  proceeding  in  a  productive  and  timely  manner,  and 
recent  (but  too  late)  intense  senior  management  involvement. 

•  The  Program  Office’s  sudden  decision  in  August  2015 
to  move  the  VSim  to  a  Naval  Air  Systems  Command 
(NAVAlR)-proposed,  government-led  Joint  Simulation 
Environment  (JSE),  will  not  result  in  a  simulation  with 
the  required  capabilities  and  fidelity  in  time  for  F-35 
IOT&E.  Without  a  high-fidelity  simulation,  the  F-35 
IOT&E  will  not  be  able  to  test  the  F-35’s  full  capabilities 
against  the  full  range  of  required  threats  and  scenarios. 
Nonetheless,  because  aircraft  continue  to  be  produced  in 
substantial  quantities  (all  of  which  will  require  some  level  of 
modifications  and  retrofits  before  being  used  in  combat),  the 
IOT&E  must  be  conducted  without  further  delay  to  evaluate 
F-35  combat  effectiveness  under  the  most  realistic  conditions 
that  can  be  obtained.  Therefore,  to  partially  compensate 

for  the  lack  of  a  simulator  test  venue,  the  JOTT  will  now 
plan  to  conduct  a  significant  number  of  additional  open-air 
flights  during  IOT&E  relative  to  the  previous  test  designs. 

In  the  unlikely  event  a  simulator  test  venue  is  available,  the 
additional  flights  would  not  be  flown. 

Suitability 

•  The  operational  suitability  of  all  variants  continues  to  be  less 
than  desired  by  the  Services  and  relies  heavily  on  contractor 
support  and  workarounds  that  would  be  difficult  to  employ  in 
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a  combat  environment.  Almost  all  measures  of  performance 
have  improved  over  the  past  year,  but  most  continue  to  be 
below  their  interim  goals  to  achieve  acceptable  suitability 
by  the  time  the  fleet  accrues  200,000  flight  hours,  the 
benchmark  set  by  the  program  and  defined  in  the  Operational 
Requirements  Document  (ORD)  for  the  aircraft  to  meet 
reliability  and  maintainability  requirements. 

-  Aircraft  fleet-wide  availability  continued  to  be  low, 
averaging  51  percent  over  12  months  ending  in 
October  2015,  compared  to  a  goal  of  60  percent. 

-  Measures  of  reliability  that  have  ORD  requirement 
thresholds  have  improved  since  last  year,  but  eight  of 
nine  measures  are  still  below  program  target  values  for 
the  current  stage  of  development,  although  two  are  within 
5  percent  of  their  interim  goal;  one — F-35B  Mean  Flight 
Hours  Between  Maintenance  Event  (Unscheduled) — is 
above  its  target  value. 

-  F-35  aircraft  spent  21  percent  more  time  than  intended 
down  for  maintenance  and  waited  for  parts  from  supply  for 
5 1  percent  longer  than  the  program  targeted.  At  any  given 
time,  from  1-in- 10  to  l-in-5  aircraft  were  in  a  depot  facility 
or  depot  status  for  major  re-work  or  planned  upgrades.  Of 
the  fleet  that  remained  in  the  field,  on  average,  only  half 
were  able  to  fly  all  missions  of  even  a  limited  capability 
set. 

-  The  amount  of  time  required  to  repair  aircraft  and  return 
them  to  flying  status  remains  higher  than  the  requirement 
for  the  system  when  mature,  but  there  has  been 
improvement  over  the  past  year. 

-  The  program  fielded  new  software  for  the  Autonomic 
Logistics  Information  System  (AL1S)  during  2015.  All 
fielded  units  transitioned  from  version  1.0.3  to  2.0.0 
between  January  and  April  2015.  Additional  increments 
were  tested — 2.0.1  and  2. 0.1.1 — which  included  software 
updates  to  correct  deficiencies  discovered  in  2.0.1. 

Version  2.0. 1.1  software  was  fielded  to  operational 
units  between  May  and  October  2015.  These  versions 
included  new  functions,  improved  interfaces,  and  fixes 
for  some  of  the  deficiencies  in  the  earlier  AL1S  versions. 
However,  many  critical  deficiencies  remain  which  require 
maintenance  personnel  to  implement  workarounds  to 
address  the  unresolved  problems. 

Live  Fire  Test  and  Evaluation  (LFT&E) 

•  The  F-35  LFT&E  program  completed  one  major  live  fire  test 
series  using  an  F-35C  variant  full-scale  structural  test  article 
(CG:0001)  with  an  installed  Pratt  and  Whitney  F135  engine. 
Preliminary  test  data  analyses: 

-  Demonstrated  the  tolerance  of  the  FI  35  initial  flight 
release  (IFR)  configured  engine  to  threat-induced  fuel 
discharge  into  the  engine  inlet 

-  Confirmed  the  expected  vulnerabilities  of  the  fuel  tank 
structure 

•  The  program  demonstrated  performance  improvements  of 
the  redesigned  fuel  tank  ullage  inerting  system  in  the  F-35B 
fuel  system  simulator  (FSS).  However,  aircraft  ground  and 


flight  tests,  designed  to  validate  the  fuel  system  simulator 
tests  and  aircraft  system  integration,  revealed  design 
deficiencies  that  require  further  hardware  and  software 
modifications. 

•  The  test  plan  to  assess  chemical  and  biological 
decontamination  of  pilot  protective  equipment  is  not 
adequate;  no  plans  have  been  made  to  test  either  the  Gen 
II  or  the  Gen  III  HMDS.  The  Program  Office  is  on  track 
to  evaluate  the  chemical  and  biological  agent  protection 
and  decontamination  systems  in  the  full-up  system-level 
decontamination  test  planned  for  FY16. 

•  The  Navy  completed  vulnerability  testing  of  the  F-35B 
electrical  and  mission  systems  to  the  electromagnetic  pulse 
(EMP). 

•  The  F-35  program  continues  to  collect  data  to  support 
the  lethality  evaluation  of  the  25  mm  x  137  mm  PGU-48 
Frangible  Armor  Piercing  (FAP)  round,  a  designated  round 
for  the  F-35A  variant,  and  the  PGU-32/U  Semi-Armor 
Piercing  High  Explosive  Incendiary-Tracer  (SAPHEI-T) 
ammunition  currently  designated  for  the  F-35B  and  F-35C 
variants. 

Air-Ship  Integration  and  Ship  Suitability 

•  The  Marine  Corps  conducted  a  suitability  demonstration 
with  six  operational  F-35B  aircraft  onboard  the  USS  Wasp 
from  May  18-29,2015. 

-  As  expected,  the  demonstration  was  not  an  operational  test 
and  could  not  demonstrate  that  the  F-35B  is  operationally 
effective  or  suitable  for  use  in  combat.  This  is  due  to  the 
following: 

■  Lack  of  production-representative  support  equipment 

■  Provision  of  extensive  supply  support  to  ensure 
replacement  parts  reached  the  ship  faster  than  would  be 
expected  in  deployed  combat  operations 

■  Incompleteness  of  the  available  maintenance  procedures 
and  technical  data,  which  required  extensive  use  of 
contractor  logistics  support 

■  Lack  of  flight  clearance  to  carry  and  employ  combat 
ordnance 

■  Lack  of  the  full  complement  of  electronic  mission 
systems  necessary  for  combat  on  the  embarked  aircraft 

■  No  other  aircraft,  and  their  associated  equipment,  that 
would  normally  be  employed  with  an  Air  Combat 
Element  (ACE)  were  present,  other  than  three  MH-60S 
rescue  helicopters 

-  The  USS  Wasp  demonstration  event  did,  however,  provide 
useful  training  for  the  Marine  Corps  and  amphibious 
Navy  with  regards  to  F-35B  operations  onboard  L-class 
ships,  and  also  provided  findings  relevant  to  the  eventual 
integration  of  the  F-35B  into  the  shipboard  environment. 
However,  aircraft  reliability  and  maintainability  were 
poor,  so  it  was  difficult  for  the  detachment  to  keep  more 
than  two  to  three  of  the  six  embarked  aircraft  in  a  flyable 
status  on  any  given  day,  even  with  significant  contractor 
assistance.  Aircraft  availability  during  the  deployment  was 
approximately  55  percent.  Around  80  percent  availability 
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would  be  necessary  to  generate  four-ship  combat 
operations  consistently  with  a  standard  six-ship  F-35B 
detachment. 

•  The  second  phase  of  F-35C  ship  suitability  testing  on  CVN 
class  carriers,  Developmental  Test  -  Two  (DT-2),  was 
conducted  from  October  2  -  10,  2015.  Ship  availability 
delayed  the  start  of  DT-2  from  the  planned  date  in 
August  2015.  The  principal  goal  of  DT-2  was  to  perform 
launch  and  recovery  of  the  F-35C  with  internal  stores  loaded. 

•  The  Navy  continues  to  work  on  numerous  air-ship 
integration  issues  including  carrier  Jet  Blast  Deflector  (JBD) 
design  limitations,  as  well  as  improving  support  equipment, 
hearing  protection,  and  firefighting  equipment. 

Cybersecurity  Testing 

•  In  accordance  with  DOT&E  and  DOD  policy,  the  JOTT 
developed  and  presented  a  cybersecurity  operational  test 
strategy  to  DOT&E  for  approval  in  February  2015.  This 
strategy  established  a  schedule  and  expectations  for 
cybersecurity  testing  of  the  JSF  air  system  through  the  end 
of  SDD  and  IOT&E  in  late  2017.  The  strategy  includes 
multiple  assessments  aligned  with  the  blocks  of  capability  as 
the  program  delivers  them  to  the  field  in  both  the  air  vehicle 
and  ALIS.  The  test  teams  will  conduct  the  assessments 

on  fielded,  operational  equipment.  All  testing  requires 
coordination  from  the  JSF  Program  Executive  Officer,  via 
an  Interim  Authority  to  Test  (1ATT).  This  testing  is  OT&E 
where  DOT&E  approves  plans  and  independently  reports 
results.  The  test  strategy,  approved  by  DOT&E,  includes 
end-to-end  testing  of  all  ALIS  components  and  the  F-35  air 
vehicle. 

•  The  JOTT  began  planning  Cooperative  Vulnerability 
and  Penetration  Assessments  (CVPAs)  and  Adversarial 
Assessments  (AAs)  of  all  ALIS  components  in  the  latest 
configuration  to  be  fielded — ALIS  2. 0.1.1 — as  well  as  the 
F-35  air  vehicle  in  the  Block  2B  configuration.  The  JOTT 
planned  a  CVPA  for  September  21  through  October  2,  2015, 
and  an  AA  from  November  9  -  20,  2015.  However,  the 
test  teams  were  not  able  to  complete  the  CVPA  as  planned 
because  the  Program  Office  failed  to  provide  an  IATT  due  to 
insufficient  understanding  of  risks  posed  to  the  operational 
ALIS  systems  by  cybersecurity  testing.  This  testing  was 
postponed  and  combined  with  an  AA,  planned  to  take  place 
in  early  November  2015.  However,  the  Program  Office 
approved  only  a  partial  IATT,  which  allowed  a  CVPA  of 
the  ALIS  components  at  Edwards  AFB,  California,  and  a 
CVPA  of  the  Operational  Central  Point  of  Entry  (CPE) — a 
major  network  hub  in  the  overall  ALIS  architecture — to 
proceed.  Although  authorized,  the  AA  for  the  CPE  was  not 
accomplished  because  the  IATT  was  approved  too  late  for 
the  AA  team  to  make  arrangements  for  the  test.  The  limited 
testing  that  was  permitted  revealed  significant  deficiencies 
that  must  be  corrected  and  highlighted  the  requirement  to 
complete  all  planned  cybersecurity  testing. 

•  Only  ALIS  components  were  planned  to  be  tested  in  these 
events  in  late  2015;  inclusion  of  the  air  vehicle  is  planned  for 
future  events.  An  end-to-end  enterprise  event,  which  links 


each  component  system,  including  the  air  vehicle,  is  required 
for  adequate  cybersecurity  operational  testing. 

Pilot  Escape  System 

•  The  program  conducted  two  sled  tests  on  the  pilot  escape 
system  in  July  and  August  2015  that  resulted  in  failures  of 
the  system  to  successfully  eject  a  manikin  without  exceeding 
load/stress  limits  on  the  manikin.  These  sled  tests  were 
needed  in  order  to  qualify  the  new  Gen  III  HMDS  for  flight 
release.  In  July  2015,  a  sled  test  on  a  103-pound  manikin 
with  a  Gen  III  helmet  at  160  knots  speed  demonstrated  the 
system  failed  to  meet  neck  injury  criteria.  The  program  did 
not  consider  this  failure  to  be  solely  caused  by  the  heavier 
Gen  III  helmet,  primarily  due  to  similarly  poor  test  results 
observed  with  the  Gen  II  helmet  on  a  103-pound  manikin 

in  2010  tests.  The  program  conducted  another  sled  test  in 
August  2015  using  a  136-pound  manikin  with  the  Gen  III 
helmet  at  160  knots.  The  system  also  failed  to  meet  neck 
injury  criteria  in  this  test.  Similar  sled  testing  with  Gen  II 
helmets  in  2010  did  not  result  in  exceedance  of  neck  loads 
for  136-pound  pilots. 

•  After  the  latter  failure,  the  Program  Office  and  Services 
decided  to  restrict  pilots  weighing  less  than  136  pounds  from 
flying  any  F-35  variant,  regardless  of  helmet  type  (Gen  II  or 
Gen  III).  Pilots  weighing  between  136  and  165  pounds  are 
considered  at  less  risk  than  lighter  weight  pilots,  but  still  at 
an  increased  risk  (compared  to  heavier  pilots).  The  level  of 
risk  was  labeled  “serious”  by  the  Program  Office  based  on 
the  probability  of  death  being  23  percent,  and  the  probability 
of  neck  extension  (which  will  result  in  some  level  of  injury) 
being  100  percent.  Currently,  the  Program  Office  and  the 
Services  have  decided  to  accept  this  level  of  risk  to  pilots 

in  this  weight  range,  although  the  basis  for  the  decision  to 
accept  these  risks  is  unknown. 

•  In  coordination  with  the  Program  Office,  the  ejection  seat 
contractor  funded  a  proof-of-concept  ejection  sled  test  in 
October  to  assess  the  utility  of  a  head  support  panel  (HSP), 
a  fabric  mesh  behind  the  pilot’s  head  and  between  the 
parachute  risers,  to  prevent  exceeding  neck  loads  during 
the  ejection  sequence  for  lighter  weight  pilots.  Based  on 
the  initial  resrdts,  the  Program  Office  and  Services  are 
considering  seat  modifications  that  would  include  the  HSP, 
but  they  may  take  at  least  a  year  to  verify  improvement  and 
install  them  onto  aircraft.  Additional  testing  and  analyses 
are  also  needed  to  determine  the  risk  to  pilots  of  being 
harmed  by  the  transparency  removal  system  (which  shatters 
the  canopy  before,  and  in  order  for,  the  seat  and  pilot  leave 
the  aircraft)  during  ejections  in  other  than  ideal,  stable 
conditions  (such  as  after  battle  damage  or  during  out-of- 
control  situations). 

•  The  program  began  delivering  F-35  aircraft  with  a 
water-activated  parachute  release  system  in  later  deliveries 
of  Lot  6  aircraft  in  2015.  This  system,  common  in  current 
fighter  aircraft  for  many  years,  automatically  jettisons  the 
parachute  when  the  pilot  enters  water  after  ejection;  in  the 
case  of  pilot  incapacitation,  an  automatic  jettisoning  of  the 
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parachute  canopy  is  essential  for  aircrew  survival.  In  June 
2012,  while  reviewing  preparations  to  begin  training  pilots  at 
Eglin  AFB,  Florida,  the  Program  Office  accepted  the  serious 
risk  of  beginning  training  without  the  water-activated  release 
system  installed  in  the  early  production  lots  of  training 
aircraft.  At  that  time,  the  Program  Office  expected  the  full 
qualification  of  the  system  to  be  completed  by  the  end  of 
2012. 

System 

•  The  F-35  JSF  program  is  a  tri-Service,  multi-national, 
single-seat,  single-engine  family  of  strike  aircraft  consisting 
of  three  variants: 

-  F-35A  Conventional  Take-Off  and  Landing  (CTOL) 

-  F-35B  Short  Take-Off/Vertical-Landing  (STOVL) 

-  F-35C  Aircraft  Carrier  Variant  (CV) 

•  It  is  designed  to  survive  in  an  advanced  threat  (year  2015  and 
beyond)  environment  using  numerous  advanced  capabilities. 
It  is  also  designed  to  have  improved  lethality  in  this 
environment  compared  to  legacy  multi-role  aircraft. 

•  Using  an  active  electronically  scanned  array  radar  and  other 
sensors,  the  F-35  is  intended  to  employ  precision-guided 
bombs  such  as  the  GBU-31/32  JDAM,  GBU-39  SDB, 

Navy  JSOW-C1,  AIM-120C  AMRAAM,  and  AIM-9X 
infrared-guided  short-range  air-to-air  missile. 


•  The  program  provides  mission  capability  in  three  increments: 

-  Block  1  (initial  training;  two  increments  were  fielded: 
Blocks  1A  and  IB) 

-  Block  2  (advanced  training  in  Block  2A  and  limited 
combat  capability  in  Block  2B) 

-  Block  3  (limited  combat  in  Block  3i  and  full  combat 
capability  in  Block  3F) 

•  The  F-35  is  under  development  by  a  partnership  of  countries: 
the  United  States,  Great  Britain,  Italy,  the  Netherlands, 
Turkey,  Canada,  Australia,  Denmark,  and  Norway. 

Mission 

•  The  Combatant  Commander  will  employ  units  equipped 
with  F-35  aircraft  to  attack  targets  during  day  or  night,  in 
all  weather  conditions,  and  in  highly  defended  areas  of  joint 
operations. 

•  The  F-35  will  be  used  to  attack  fixed  and  mobile  land  targets, 
surface  units  at  sea,  and  air  threats,  including  advanced 
aircraft  and  cruise  missiles. 

Major  Contractor 

Lockheed  Martin,  Aeronautics  Division  -  Fort  Worth,  Texas 


Test  Strategy,  Planning,  and  Resourcing 

•  The  Program  Office  continues  to  plan  for  a  start  of  IOT&E 
in  August  2017,  three  months  after  the  program’s  planned 
completion  of  developmental  flight  test  in  May  2017,  or 
one  month  later  than  the  recently  briefed  date  of  July  2017. 

In  the  intervening  three  months,  the  program  must  complete 
all  the  analyses  and  certification  requirements  to  allow  final 
preparations  for  IOT&E  to  begin.  There  are  clear  indications 
that  it  is  no  longer  possible  to  meet  the  requirements  to  start 
an  adequate  IOT&E  at  that  time.  Specifically: 

-  The  program’s  joint  test  plans  for  Block  3F  mission 

systems  testing  contain  more  testing  than  can  be  completed 
by  May  2017,  which  is  the  planned  end  of  Block  3F  flight 
test,  according  to  the  most  recent  program  schedule. 

Even  extending  until  the  end  of  July  2017  to  compete 
System  Development  and  Demonstration  (SDD)  flight 
test  is  not  realistic.  Instead,  the  program  will  likely  not 
finish  Block  3F  development  and  flight  testing  prior  to 
January  2018,  based  on  the  following: 

■  Continuing  a  six  test  point  per  flight  accomplishment 
rate,  which  is  equal  to  the  CY15  rate  observed  through 
the  end  of  November 

■  Continuing  a  flight  rate  of  6.8  flights  per  month  with  the 
6  mission  systems  developmental  test  aircraft  assigned 
to  Edwards  AFB,  as  was  achieved  through  the  end  of 
November  2015,  exceeding  the  planned  rate  of  6  flights 
per  month  (if  the  flight  rate  deteriorates  to  the  planned 
rate  of  6  flights  per  month,  then  testing  will  not  complete 
until  May  2018) 


■  Completing  the  full  Block  3F  mission  systems  test  plan 
(i.e.,  all  original  7,230  baseline  and  budgeted  non¬ 
baseline  test  points  in  the  Block  3F  joint  test  plan) 

■  Continuing  the  CY15  discovery  rate  of  5  percent 

-  Based  on  these  projected  completion  dates  for  Block  3F 
developmental  testing,  IOT&E  would  not  start  earlier  than 
August  2018.  The  program  could,  as  has  been  the  case  in 
testing  previous  software  increments,  determine  that  test 
points  in  the  plan  are  no  longer  required  for  the  Block  3F 
fleet  release.  However,  the  program  will  need  to  ensure 
that  deleting  and/or  deferring  testing  from  Block  3F  before 
the  end  of  SDD  and  the  start  of  IOT&E  does  not  result  in 
increasing  the  likelihood  of  discovery  in  IOT&E  or  affect 
the  assessment  of  mission  capability.  Whatever  capability 
the  program  determines  as  ready  for  IOT&E  will  undergo 
the  same  realistic  and  rigorous  combat  mission-focused 
testing  as  a  fully  functioning  system. 

-  The  48  Block  3F  developmental  test  weapons  delivery 
accuracy  (WDA)  events  in  the  approved  Test  and 
Evaluation  Master  Plan  (TEMP),  plus  two  test  events 
deferred  from  Block  2B,  will  not  be  accomplished  by  the 
planned  date  of  May  2017,  according  to  the  program’s 
official  schedule,  nor  by  July  2017,  a  more  recently 
briefed  date  for  the  completion  of  SDD  flight  test,  unless 
the  program  is  able  to  significantly  increase  their  historic 
WDA  completion  rate.  In  order  to  meet  the  schedule 
requirements  for  weapon  certification,  the  Program  Office 
has  identified  10  WDA  events  for  the  F-35  A  and  5  events 
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for  the  F-35B  and  F-35C  that  must  be  accomplished  during 
Block  3F  developmental  testing.  The  program  plans  to 
accomplish  the  remaining  33  events  as  schedule  margin 
allows. 

-  Modifying  the  fleet  of  operational  test  aircraft  to  the 
required  production-representative  Block  3F  configuration, 
with  the  TEMP-required  instrumentation  capability,  will 
not  be  complete  before  August  2017. 

-  The  Program  Office  did  not  put  the  Block  3F  Verification 
Simulation  (VSim)  development  on  contract  in  early 
2015,  as  was  needed  in  order  to  complete  development 
for  IOT&E.  The  Program  Office  decided  instead  to 
move  from  VSim  to  the  Joint  Simulation  Environment 
(JSE),  which  will  result  in  a  fully  verified,  validated,  and 
accredited  simulator  not  being  ready  in  time  for  IOT&E. 

•  Comparison  testing  provides  insight  into  the  capabilities 
available  from  new  weapon  systems  relative  to  the  legacy 
systems  they  replace.  Since  the  Department  plans  to  retire 
a  large  portion  of  its  tactical  aircraft  inventory  and  replace 
them  over  time  with  the  F-35,  comparison  testing  will 

be  a  part  of  the  Block  3F  IOT&E.  The  JSF  Operational 
Test  Team  (JOTT),  in  coordination  with  DOT&E  staff, 
began  to  develop  test  plans  for  IOT&E,  which  will  include 
comparisons  of  the  F-35  with  the  A- 10  in  the  Close 
Air  Support  role  and  with  the  F-16C  (Block  50)  in  the 
Suppression  of  Enemy  Air  Defense/Destruction  of  Enemy 
Air  Defenses  (SEAD/DEAD)  mission  area.  Comparison 
testing  involving  other  strike  aircraft  is  under  consideration 
by  the  JOTT  and  DOT&E. 

•  JSF  follow-on  development  will  integrate  additional 
capabilities  in  Block  4,  address  deferrals  from  Block  3F  to 
Block  4,  and  correct  deficiencies  discovered  during  Block  3F 
development  and  IOT&E. 

-  The  program  plans  to  complete  Block  3F  software 
development  in  2016  and  flight  testing  in  early  2017. 

The  next  planned  software  delivery  will  be  a  Block  4 
build  in  2020,  creating  a  four  year  gap  between  planned 
software  releases.  Considering  the  large  number  of  open 
deficiencies  documented  from  Blocks  2B  and  3i  testing, 
the  ongoing  discovery  of  deficiencies  during  Block  3F 
testing,  and  the  certainty  of  more  discoveries  from  IOT&E, 
the  program  needs  to  plan  for  additional  Block  3F  software 
builds  and  follow-on  testing  prior  to  2020. 

-  As  has  been  the  case  with  the  F-22,  the  F-35  program 
will  remain  on  DOT&E  oversight  during  follow-on 
development  and  therefore  must  plan  for  and  fund  an 
associated  formal  OT&E  of  each  Block  4  increment  prior 
to  release  to  operational  units. 

•  The  program  has  proposed  a  “block  buy”  combining  three 
production  lots  comprising  as  many  as  270  U.S.  aircraft 
purchases  to  gain  near-term  savings.  A  commitment  to  the 
“block  buy”  could  be  necessary  before  IOT&E  is  complete. 

In  that  case,  entering  a  “block  buy”  would  raise  the  following 
questions: 

-  Is  the  F-35  program  sufficiently  mature  to  commit  to 
the  “block  buy?”  The  program  continues  to  discover 


significant  problems  during  developmental  testing  that, 
if  not  addressed  with  corrections  or,  in  some  cases, 
labor-intensive  workarounds,  will  adversely  affect 
the  operational  effectiveness  and  suitability  of  all 
three  variants;  these  deficiencies  need  to  be  corrected 
before  the  system  is  used  in  combat.  To  date,  the  rate  of 
deficiency  correction  has  not  kept  pace  with  the  discovery 
rate.  Examples  of  well-known  significant  problems 
include  the  immaturity  of  the  Autonomic  Logistics 
Information  System  (ALIS),  Block  3F  avionics  instability, 
and  several  reliability  and  maintainability  problems 
with  the  aircraft  and  engine.  Much  of  the  most  difficult 
and  time-consuming  developmental  testing,  including 
approximately  50  complex  WDA  events,  remains  to 
be  completed.  Hence,  new  discoveries,  some  of  which 
could  further  affect  the  design  or  delay  the  program,  are 
likely  to  occur  throughout  the  time  the  Department  could 
commit  to  the  “block  buy.”  Recent  discoveries  that  require 
design  changes,  modifications,  and  regression  testing 
include  the  ejection  seat  for  safe  separation,  wing  fuel  tank 
over-pressurization,  and  the  life-limitations  of  the  F-35B 
bulkhead.  For  these  specific  reasons  and  others,  further 
program  delays  are  likely. 

-  Is  it  appropriate  to  commit  to  a  “block  buy”  given  that 
essentially  all  the  aircraft  procured  thus  far  require 
modifications  to  be  used  in  combat?  Although  still 
officially  characterized  as  low-rate,  F-35  production  rates 
are  already  high.  Despite  the  problems  listed  above,  F-35 
production  rates  have  been  allowed  to  steadily  increase  to 
large  rates,  well  prior  to  the  IOT&E  and  official  Full-Rate 
Production  (FRP)  decision.  Due  to  this  concurrency  of 
development  and  production,  approximately  340  aircraft 
will  be  produced  by  FY 1 7  when  developmental  testing  is 
currently  planned  to  end,  and  over  500  aircraft  by  FY19 
when  IOT&E  will  likely  end  and  the  FRP  milestone 
decision  should  occur.  These  aircraft  will  require  a 
still-to-be-determined  list  of  modifications  in  order  to 
provide  full  Block  3F  combat  capability.  However,  these 
modifications  may  be  unaffordable  for  the  Services  as  they 
consider  the  cost  of  upgrading  these  early  lots  of  aircraft 
while  the  program  continues  to  increase  production  rates  in 
a  fiscally-constrained  environment.  This  may  potentially 
result  in  left-behind  aircraft  with  significant  limitations  for 
years  to  come. 

-  Would  committing  to  a  “block  buy”  prior  to  the 
completion  of  IOT&E  provide  the  contractor  with  needed 
incentives  to  fix  the  problems  already  discovered,  as  well 
as  those  certain  to  be  discovered  during  IOT&E?  Would 
it  be  preferred — and  would  it  provide  a  strong  incentive  to 
fix  problems  and  deliver  fully  combat-capable  aircraft — to 
make  the  “block  buy,”  as  well  as  any  additional  increases 
in  the  already  high  annual  production  rate,  contingent 
upon  successful  completion  of  IOT&E?  Similarly, 
would  the  “block  buy”  also  be  consistent  with  the  “fly 
before  you  buy”  approach  to  acquisition  advocated  by 
the  Administration,  as  well  as  with  the  rationale  for  the 
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operational  testing  requirements  specified  in  Title  10  U.S. 
Code? 

•  This  report  includes  assessments  of  the  progress  of  testing 
to  date,  including  developmental  and  operational  testing 
intended  to  verify  performance  prior  to  the  start  of  IOT&E. 

-  For  developmental  flight  testing,  the  program  creates  plans 
by  identifying  specific  test  points  (discrete  measurements 
of  performance  under  specific  flight  test  conditions)  for 
accomplishment  in  order  to  determine  capabilities  as  being 
compliant  with  contract  specifications. 

■  Baseline  test  points  refer  to  points  in  the  test  plans 
that  must  be  accomplished  in  order  to  evaluate  if 
performance  meets  contract  specifications. 

■  Non-baseline  test  points  are  accomplished  for  various 
reasons.  Program  plans  include  a  budget  for  some  of 
these  points  within  the  capacity  of  flight  test  execution. 
The  following  describes  non-baseline  test  points. 

»  Development  points  are  test  points  required  to  “build 
up”  to,  or  prepare  for,  the  conditions  needed  for 
specification  compliance  (included  in  non-baseline 
budgeted  planning  in  CY15). 

»  Regression  points  are  test  points  flown  to  ensure 
that  new  software  does  not  introduce  discrepancies 
as  compared  to  previous  software  (included  in 
non-baseline  budgeted  planning  in  CY15). 

»  Discovery  points  are  test  points  flown  to  investigate 
root  causes  or  characterize  deficiencies  so  that  the 
program  can  design  fixes  (not  included  in  planning  in 
CY15). 


-  As  the  program  developed  plans  for  allocating  test 
resources  against  test  points  in  CY15,  the  program 
included  a  larger  budget  for  non-baseline  test  points 
(development  and  regression  points)  for  all  test  venues 
(i.e.,  each  variant  of  flight  sciences  and  mission  systems). 
For  CY15  mission  systems  testing,  planners  budgeted 

an  additional  45  percent  of  the  number  of  planned 
baseline  test  points  for  non-baseline  test  purposes  (e.g., 
development  and  regression  points).  In  this  report,  growth 
in  test  points  refers  to  points  flown  in  addition  to  the 
planned  amount  of  baseline  and  budgeted  non-baseline 
points  (e.g.,  discovery  points  and  any  other  added  testing 
not  originally  included  in  the  formal  test  plan).  The 
program  allocates  budgeted  non-baseline  test  points  in 
specific  quantities  to  test  categories  (i.e.,  variant  flight 
science,  Block  2B,  3i,  and  3F  mission  systems). 

-  The  need  to  budget  for  non-baseline  test  points  in  the 
CY15  plan  is  a  result  of  the  limited  maturity  of  capability 
in  the  early  versions  of  mission  systems  software.  In 
CY15,  when  the  first  versions  of  Block  3F  software  were 
planned  to  be  introduced  to  flight  testing,  limited  baseline 
test  points  could  be  completed  and  development  points 
would  be  the  majority  of  the  type  of  points  flown.  Also, 
as  three  versions  of  Block  3F  software  were  planned  to  be 
introduced  to  flight  testing  in  CY15,  the  test  centers  would 
need  to  accomplish  a  large  number  of  regression  points. 

-  Cumulative  SDD  test  point  data  in  this  report  refer  to  the 
total  progress  towards  completing  development  at  the  end 
of  SDD. 


TEST  FLIGHTS  (AS  OF  NOVEMBER  2015) 

All  Testing 

Flight  Sciences 

Mission 

Systems 

All  Variants 

F-35A 

F-35B 

F-35C 

201 5  Actual 

1,193 

188 

283 

270 

452 

2015  Planned 

1,281 

231 

311 

256 

483 

Difference  from  Planned 

7.4% 

22.9% 

9.9% 

-5.2% 

6.9% 

Cumulative  Planned 

6,242 

1,489 

1,844 

1,188 

1,721 

Cumulative  Actual 

6,416 

1,466 

1,893 

1,193 

1,864 

Difference  from  Planned 

2.8% 

-1.5% 

2.7% 

0.4% 

8.3% 

PriortoCY15  Planned 

5,049 

1,301 

1,561 

918 

1,269 

Prior  to  CY1 5  Actual 

5,135 

1,235 

1,582 

937 

1,381 
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TEST  POINTS  (AS  OF  NOVEMBER  201 5) 


All  Testing 

Flight  Sciences1 

Mission  Systems 

All  Variants 

F-35A 

F-35B 

F-35C 

Block 

2B 

Block 

3i 

Block 

3F 

Budgeted 

Non- 

Baseline2 

Other3 

Block  3F 
Baseline 

Budgeted 

Non- 

Baseline2 

Block  3F 
Baseline 

Budgeted 

Non- 

Baseline2 

Block  3F 
Baseline 

Budgeted 

Non- 

Baseline2 

201 5  Test  Points 
Planned 
(by  type) 

8,673 

1,221 

113 

2,181 

211 

1,819 

130 

143 

514 

575 

1,097 

669 

201 5  Test  Points 
Accomplished  (by 
type) 

8,011 

1,196 

62 

2,003 

191 

1,910 

59 

160 

469 

674 

834 

453 

Difference  from 
Planned 

-7.6% 

-2.0% 

-45.1% 

-8.2% 

-9.5% 

5.0% 

-54.6% 

11.9% 

-8.8% 

1 7.2% 

-24.0% 

-32.3% 

Points  Added 

Beyond  Budgeted 
Non-Baseline 
(Growth  Points) 

457 

0 

0 

0 

93 

364 

0 

0 

0 

Test  Point  Growth 

Percentage 

(Growth 

Points/Test  Points 
Accomplished) 

5.7% 

0.0% 

0.0% 

0.0% 

58.1% 

77.6% 

0.0% 

0.0% 

0.0% 

Total  Points (by 
type)  Accomplished 
in  201 54 

8,468 

1,258 

2,194 

1,969 

253 

833 

674 

834 

453 

Cumulative  Data 

Cumulative  SDD 
Planned  Baseline5 

43,61 1 

10,919 

1 3,995 

10,650 

6,232 

699 

575 

N/A 

541 

Cumulative  SDD 
Actual  Baseline 

43,528 

10,978 

1 3,835 

10,729 

5,933 

660 

674 

N/A 

719 

Difference  from 
Planned 

-0.2% 

0.5% 

-1.1% 

0.7% 

-4.8% 

-5.6% 

1 7.2% 

N/A 

32.9% 

Estimated  Test 
Baseline  Points 
Remaining 

12,905 

1,597 

3,250 

2,428 

0 

0 

4,841 

N/A 

789 

Estimated  Non- 
Baseline  Test  Points 
Remaining 

2,175 

139 

443 

270 

0 

0 

1,323 

N/A 

0 

1 .  Flight  sciences  test  points  for  CY1 5  are  shown  only  for  Block  3F.  Block  2B  Flight  Sciences  testing  was  completed  in  CY1 4  for  F-35A,  May  201 5  for  F-35B,  and  January  201 5  for  F-35C.  Cumulative 

numbers  include  all  previous  flight  science  activity. 

2.  These  points  account  for  planned  development  and  regression  test  points  built  into  the  201 5  plan;  additional  points  are  considered  "growth". 

3.  Represents  mission  systems  activity  not  directly  associated  with  Block  capability  (e.g.,  radar  cross  section  characterization  testing,  test  points  to  validate  simulator). 

4.  Total  Points  Accomplished  =  201 5  Baseline  Accomplished  +  Added  Points 

5.  SDD  -  System  Design  and  Development 

F-35A  Flight  Sciences 

Flight  Test  Activity  with  AF-1,  AF-2,  and  AF-4  Test  Aircraft 

•  F-35A  flight  sciences  testing  focused  on: 

-  Internal  gun  testing 

-  Flight  envelope  expansion  with  external  weapons  required 
for  Block  3F  weapons  capability 

-  Air  refueling  qualification  with  Italian  and  Australian 
tanker  aircraft 

-  Testing  to  mitigate  fuel  system  over-pressurization 
conditions  caused  by  fuel  and  On-Board  Inert  Gas 
Generation  System  (OBIGGS)  gas  pressure  stacking 
within  the  system 


F-35A  Flight  Sciences  Assessment 

•  Through  the  end  of  November,  the  test  team  flew  23  percent 
more  flights  than  planned  (231  flown  versus  188  planned), 
but  was  2  percent  behind  the  plan  for  Block  3F  baseline  test 
point  completion  (1,196  test  points  accomplished  versus 
1,221  planned).  By  the  end  of  November  2015,  the  test 
team  flew  an  additional  62  test  points  for  regression  of 
new  air  vehicle  software  (which  were  part  of  the  budgeted 
non-baseline  test  points  allocated  for  the  year)  and  238 
points  for  air  refueling  qualification  with  partner  nation 
tanker  aircraft  (these  points  are  not  included  in  the  table  of 
test  flights  and  test  points  above).  All  F-35A  flight  sciences 
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testing  accomplished  in  CY15  was  relevant  to  Block  3F 
requirements. 

All  Block  2B  flight  sciences  test  points  were  completed  in 
CY14  and  provided  the  basis  for  the  F-35A  Block  2B  fleet 
release  to  the  training  and  operational  units  in  August  2015. 
The  Block  2B  flight  sciences  test  points  also  provided  the 
basis  for  Block  3i  initial  flight  clearances  needed  for  Lot  6 
and  Lot  7  production  aircraft  delivered  in  CY15.  There  is  no 
additional  flight  envelope  provided  by  Block  3i  compared  to 
Block  2B. 

The  following  details  discoveries  in  F-35A  flight  sciences 
testing: 

-  Testing  to  characterize  the  thermal  environment  of  the 
weapons  bays  demonstrated  that  temperatures  become 
excessive  during  ground  operations  in  high  ambient 
temperature  conditions  and  in-flight  under  conditions 
of  high  speed  and  at  altitudes  below  25,000  feet.  As  a 
result,  during  ground  operations,  fleet  pilots  are  restricted 
from  keeping  the  weapons  bay  doors  closed  for  more 
than  10  cumulative  minutes  prior  to  take-off  when 
internal  stores  are  loaded  and  the  outside  air  temperature 
is  above  90  degrees  Fahrenheit.  In  flight,  the  10-minute 
restriction  also  applies  when  flying  at  airspeeds  equal  to 
or  greater  than  500  knots  at  altitudes  below  5,000  feet; 

550  knots  at  altitudes  between  5,000  and  15,000  feet;  and 
600  knots  at  altitudes  between  15,000  and  25,000  feet. 
Above  25,000  feet,  there  are  no  restrictions  associated 
with  the  weapons  bay  doors  being  closed,  regardless 

of  temperature.  The  time  limits  can  be  reset  by  flying 
10  minutes  outside  of  the  restricted  conditions  (i.e.,  slower 
or  at  higher  altitudes).  This  will  require  pilots  to  develop 
tactics  to  work  around  the  restricted  envelope;  however, 
threat  and/or  weather  conditions  may  make  completing  the 
mission  difficult  or  impossible  using  the  work  around. 

-  Testing  to  characterize  the  vibrational  and  acoustic 
environment  of  the  weapons  bays  demonstrated  that 
stresses  induced  by  the  environment  were  out  of  the  flight 
qualification  parameters  for  both  the  AIM- 120  missile  and 
the  flight  termination  system  (telemetry  unit  attached  to  the 
missile  body  required  to  satisfy  range  safety  requirements 
for  terminating  a  live  missile  in  a  flight  test).  This  resulted 
in  reduced  service  life  of  the  missile  and  potential  failure 
of  the  telemetered  missile  termination  system  required  for 
range  safety. 

-  Deficiencies  in  the  sequencing  of  release  commands  for 
the  Small  Diameter  Bomb  (SDB)  from  the  Bomb  Rack 
Unit-61,  which  provides  the  interface  between  the  SDB 
and  the  aircraft,  were  discovered  in  the  lab  and  verified  in 
aircraft  ground  testing.  The  program  will  assess  software 
corrections  to  address  these  deficiencies  in  future  flight 
testing. 

-  Mechanical  rubbing  between  the  gun  motor  drive  and  the 
wall  of  the  gun  bay  was  discovered  during  initial  ground 
testing  of  the  gun  on  the  AF-2  test  aircraft,  requiring 
structural  modifications  to  the  bay  and  alterations  to  the 
flow  of  cooling  air  and  venting  of  gun  gasses. 


-  Under  certain  flight  conditions,  air  enters  the  siphon 
fuel  transfer  line  and  causes  the  pressure  in  the  siphon 
fuel  tank  to  exceed  allowable  limits  in  all  variants.  As 
a  result,  the  program  imposed  an  aircraft  operating 
limitation  (AOL)  on  developmental  test  aircraft  limiting 
maneuvering  flight  for  each  variant  (e.g.  “g”  load  during 
maneuvering).  F-35A  developmental  test  aircraft  with  the 
most  recent  fuel  tank  ullage  inerting  system  modifications 
are  limited  to  3.8  g’s  when  the  aircraft  is  fully  fueled.  The 
allowable  g  increases  as  fuel  is  consumed  and  reaches 

the  full  Block  2B  7.0  g  envelope  (a  partial  envelope 
compared  to  full  Block  3F)  once  total  fuel  remaining 
is  10,213  pounds  or  less,  or  roughly  55  percent  of  full 
fuel  capacity,  for  developmental  test  aircraft  with  test 
control  team  monitoring  (through  instrumentation)  of 
the  fuel  system.  For  developmental  test  aircraft  without 
fuel  system  monitoring,  the  full  Block  2B  7.0  g  envelope 
becomes  available  at  9,243  pounds,  or  roughly  50  percent 
of  full  fuel  capacity.  Flight  testing  to  clear  the  F-35A  to 
the  full  Block  3F  9.0  g  envelope,  planned  to  be  released 
in  late  2017,  is  being  conducted  with  developmental  test 
aircraft  with  fuel  system  monitoring.  Fleet  F-35A  aircraft 
are  limited  to  3.0  g’s  when  fully  fueled  and  the  allowable 
g  is  increased  as  fuel  is  consumed,  reaching  the  full 
Block  2B  7.0  g  envelope  when  approximately  55  percent 
of  full  fuel  capacity  is  reached.  The  program  modified 
the  AF-4  test  aircraft  in  October  and  November  with  the 
addition  of  a  relief  line,  controlled  by  a  solenoid  valve,  to 
vent  the  affected  siphon  tanks,  and  a  check  valve  on  the 
inert  gas  line  feeding  the  tanks.  The  test  team  completed 
testing  of  the  modified  design  in  late  November  2015; 
the  results  are  under  review.  Until  relieved  of  the  g 
restrictions,  operational  units  will  have  to  adhere  to  a 
reduced  maneuvering  (i.e.,  less  “g  available”)  envelope  in 
operational  planning  and  tactics;  for  example,  managing 
threat  engagements  and  escape  maneuvers  when  in 
the  restricted  envelope  where  less  g  is  available.  This 
restriction  creates  an  operational  challenge  when  forward 
operating  locations  or  air  refueling  locations  are  close 
to  the  threat/target  arena,  resulting  in  high  fuel  weights 
during  engagements. 

-  Testing  of  operational  “dog-fighting”  maneuvers  showed 
that  the  F-35A  lacked  sufficient  energy  maneuverability  to 
sustain  an  energy  advantage  over  fourth  generation  fighter 
aircraft.  Test  pilots  flew  1 7  engagements  between  an 
F-35A  and  an  F-16D,  which  was  configured  with  external 
fuel  tanks  that  limited  the  F-16D  envelope  to  7.0  g’s. 

The  F-35A  remained  at  a  distinct  energy  disadvantage 
on  every  engagement.  Pitch  rates  were  also  problematic, 
where  full  aft  stick  maneuvers  would  result  in  less  than 
full  permissible  g  loading  (i.e.,  reaching  6.5  g  when  limit 
was  9.0  g),  and  subsequent  rapid  loss  of  energy.  The  slow 
pitch  rates  were  observed  at  slower  speeds — in  a  gun 
engagement,  for  example — that  restricted  the  ability  of  an 
F-35A  pilot  to  track  a  target  for  an  engagement. 
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•  The  program  completed  the  final  weight  assessment  of  the 
F-35A  air  vehicle  for  contract  specification  compliance 

in  April  with  the  weighing  of  AF-72,  a  Lot  7  production 
aircraft.  Actual  empty  aircraft  weight  was  28,999  pounds, 
372  pounds  below  the  planned  not-to-exceed  weight  of 
29,371  pounds.  The  program  has  managed  the  weight 
growth  of  the  F-35A  air  vehicle  with  no  net  weight  growth 
for  the  76  months  preceding  the  final  weight  assessment. 
Weight  management  of  the  F-35A  is  important  for  meeting 
performance  requirements  and  structural  life  expectations. 
The  program  will  need  to  continue  disciplined  management 
of  the  actual  aircraft  weight  beyond  the  contract  specification 
as  further  discoveries  during  the  remainder  of  SDD  may  add 
weight  and  res  Lilt  in  performance  degradation  that  would 
adversely  affect  operational  capability. 

F-35B  Flight  Sciences 

Flight  Test  Activity  with  BF- 1,  BF-2,  BF-3,  BF-4,  and  BF-5  Test 

Aircraft 

•  F-35B  flight  sciences  focused  on: 

-  Completing  Block  2B  flight  envelope  testing  by  the  end  of 
May 

-  Flight  envelope  expansion  with  external  weapons, 
including  Paveway  IV  bombs,  required  for  Block  3F 
weapons  capability 

-  Testing  to  characterize  and  mitigate  fuel  system 
over-pressurization  conditions  caused  by  fuel  and 
OBIGGS  gas  pressure  stacking  within  the  system 

-  Air  refueling  testing,  including  low  altitude  air  refueling 
with  KC-130  tanker  aircraft 

-  Testing  of  control  authority  during  landings  in  crosswind 
conditions,  both  with  and  without  external  stores 

F-35B  Flight  Sciences  Assessment 

•  Through  the  end  of  November,  the  test  team  was  able 
to  fly  10  percent  more  flights  than  planned  (311  flown 
versus  283  planned),  but  accomplished  8  percent  less  than 
the  planned  Block  3F  baseline  test  points  (2,003  points 
accomplished  versus  2,181  planned).  The  team  flew  an 
additional  191  test  points  for  regression  of  new  air  vehicle 
software,  which  were  part  of  the  budgeted  non-baseline 
points  planned  for  CY15.  The  team  also  completed  four  test 
points  needed  to  complete  the  Block  2B  flight  envelope.  The 
program  also  declared  that  23  planned  Block  2B  baseline 
points  were  no  longer  required. 

•  The  following  details  discoveries  in  F-35B  flight  sciences 
testing: 

-  Testing  to  characterize  the  thermal  environment  of  the 
weapons  bays  demonstrated  that  temperatures  become 
excessive  during  ground  operations  in  high  ambient 
temperature  conditions.  As  a  result,  during  ground 
operations,  fleet  pilots  are  restricted  from  keeping  the 
weapons  bay  doors  closed  for  more  than  10  cumulative 
minutes  prior  to  take-off  when  internal  stores  are  loaded 
and  the  outside  air  temperature  is  above  90  degrees 
Fahrenheit.  Time  with  the  weapons  bay  doors  closed  in 
flight  is  currently  not  restricted. 


-  Under  certain  flight  conditions,  air  can  enter  the  siphon 
fuel  transfer  line  and  cause  the  pressure  in  the  siphon 
fuel  tanks  to  exceed  allowable  limits  in  all  variants. 

As  a  result,  the  program  imposed  an  aircraft  operating 
limitation  (AOL)  on  developmental  test  aircraft  limiting 
maneuvering  flight  for  each  variant.  The  program 
implemented  a  partial  mitigation  in  software  on  the 
F-35B.  For  F-35B  developmental  aircraft  with  the  most 
recent  fuel  tank  ullage  inerting  system  modifications, 
the  AOL  limits  maneuvers  to  5.0  g’s  when  the  aircraft 

is  fully  fueled,  but  the  allowable  g  increases  as  fuel  is 
consumed.  The  full  Block  2B  5.5  g  envelope  (a  partial 
envelope  compared  to  Block  3F)  is  available  once  total 
fuel  remaining  is  approximately  13,502  pounds,  or  roughly 
96  percent  fuel  remaining  for  developmental  test  aircraft 
with  ground  station  monitoring  of  the  fuel  system,  and 
7,782  pounds  or  less,  or  roughly  56  percent  fuel  remaining 
for  developmental  test  aircraft  without  monitoring.  Flight 
testing  to  clear  the  F-35B  to  the  full  Block  3F  7.0  g 
envelope,  planned  to  be  released  in  late  2017,  is  being 
conducted  with  developmental  test  aircraft  with  fuel 
system  monitoring.  Fleet  F-35B  aircraft  are  limited  to 
3.0  g’s  when  fully  fueled  and  the  allowable  g  is  increased 
as  fuel  is  consumed,  reaching  the  full  Block  2B  envelope 
of  5.5  g’s  at  roughly  63  percent  of  fuel  remaining.  The 
program  has  successfully  developed  and  tested  a  hardware 
change  on  the  F-35B  to  correct  the  overpressure  problem 
involving  the  addition  of  a  relief  line  controlled  by  a  check 
valve  to  vent  the  affected  siphon  tanks.  Once  installed 
in  fleet  aircraft,  the  relief  line  and  check  valve  will 
prevent  the  pressure  in  the  siphon  tanks  from  exceeding 
the  allowable  limits.  Until  the  F-35B  aircraft  have  the 
modification  that  relieves  the  g  restrictions,  operational 
units  will  have  to  adhere  to  a  reduced  maneuvering  (i.e., 
less  “g  available”)  envelope  in  operational  planning  and 
tactics;  for  example,  managing  threat  engagements  and 
escape  maneuvers  when  in  the  restricted  envelope  where 
less  g  is  available.  This  restriction  creates  an  operational 
challenge  when  forward  operating  locations  or  air 
refueling  locations  are  close  to  the  threat/target  arena. 

-  Air  refueling  with  strategic  tankers  (KC-135  and  KC-10) 
was  restricted  to  use  of  centerline  boom-to-drogue  adapter 
(BDA)  refueling  only.  Refueling  from  tanker  wing  pods 
was  prohibited  due  to  response  anomalies  from  the  hose 
and  reel  assemblies  and  the  F-35B  aircraft  with  the  air 
refueling  receptacle  deployed. 

•  Weight  management  of  the  F-35B  aircraft  is  critical  to 
meeting  the  Key  Performance  Parameters  (KPPs)  in  the 
Operational  Requirements  Document  (ORD),  including 
the  vertical  lift  bring-back  requirement,  which  will  be 
evaluated  during  IOT&E.  This  Key  Performance  Parameter 
(KPP)  requires  the  F-35B  to  be  able  to  fly  an  operationally 
representative  profile  and  recover  to  the  ship  with  the 
necessary  fuel  and  balance  of  unexpended  weapons  (two 
1,000-pound  bombs  and  two  AIM- 120  missiles)  to  safely 
conduct  a  vertical  landing. 
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-  The  program  completed  the  final  weight  assessment  of  the 
F-35B  air  vehicle  for  contract  specification  compliance  in 
May  2015  with  the  weighing  of  BF-44,  a  Lot  7  production 
aircraft.  Actual  empty  aircraft  weight  was  32,442  pounds, 
only  135  pounds  below  the  planned  not-to-exceed  weight 
of  32,577  pounds  and  307  pounds  (less  than  1  percent) 
below  the  objective  vertical  lift  bring-back  not-to-exceed 
weight  of  32,749  pounds. 

-  The  program  will  need  to  continue  disciplined 
management  of  weight  growth  for  the  F-35B,  especially 
in  light  of  the  small  weight  margin  available  and  the 
likelihood  of  continued  discovery  through  the  remaining 
two  years  of  development  in  SDD. 

F-35C  Flight  Sciences 

Flight  Test  Activity  with  CF- 1,  CF-2,  CF-3,  and  CF-5  Test  Aircraft 

•  F-35C  flight  sciences  focused  on: 

-  Completing  Block  2B  testing  by  the  end  of  January  2015 

-  Ship  suitability  testing  in  preparation  for  the  next  set 
of  ship  trials  (DT-2),  originally  planned  for  August,  but 
slipped  to  October  2015  due  to  carrier  availability 

-  Flight  envelope  expansion  with  external  weapons,  required 
for  Block  3F  weapons  capability 

-  Testing  with  wing  spoilers  to  reduce  the  adverse  effects 
of  transonic  roll  off  in  the  portions  of  the  flight  envelope 
where  it  occurs 

-  High  angle  of  attack  testing 

-  Testing  of  control  authority  during  landings  in  crosswind 
conditions,  both  with  and  without  external  stores 

-  Testing  of  landings  on  wet  runways  and  the  effectiveness 
of  anti-skid  braking  procedures 

-  Air  refueling  testing 

-  Initial  testing  of  the  Joint  Precision  Approach  and  Landing 
System 

F-35C  Flight  Sciences  Assessment 

•  Through  the  end  of  November,  the  test  team  flew  5  percent 
less  than  planned  flights  (256  flown  versus  270  planned), 
but  accomplished  5  percent  more  than  the  planned  Block  3F 
baseline  test  points  (1,910  points  accomplished  versus 
1,819  planned).  The  team  flew  an  additional  59  test  points 
for  regression  of  new  software,  which  were  part  of  the 
budgeted  non-baseline  points  planned  for  the  year.  With 
the  exception  of  three  high  angle  of  attack  test  points  in 
January  for  the  Block  2B  envelope,  all  testing  in  CY15 
supported  Block  3F  testing  requirements. 

•  The  following  details  discoveries  in  F-35C  flight  sciences 
testing: 

-  Under  certain  flight  conditions,  air  can  enter  the  siphon 
fuel  transfer  line  and  cause  the  pressure  in  the  siphon 
fuel  tank  to  exceed  allowable  limits  in  all  variants.  The 
program  imposed  an  AOL  on  developmental  test  aircraft, 
limiting  maneuvering  flight  for  each  variant.  On  F-35C 
developmental  test  aircraft  with  the  most  recent  fuel 
tank  ullage  inerting  system  modifications,  the  AOL 
limits  maneuvers  to  4.0  g’s  when  the  aircraft  is  fully 


fueled  and  the  allowable  g  increases  as  fuel  is  consumed. 
The  full  Block  2B  6.0  g  envelope  (a  partial  envelope 
compared  to  Block  3F)  is  available  with  18,516  pounds 
or  roughly  93  percent  fuel  remaining  for  developmental 
test  aircraft  with  test  control  team  monitoring  (through 
instrumentation)  of  the  fuel  system,  and  8,810  pounds 
or  roughly  40  percent  fuel  remaining  for  developmental 
test  aircraft  without  monitoring.  Flight  testing  to  clear 
the  F-35C  to  the  full  Block  3F  7.5  g  envelope,  planned 
to  be  released  in  late  2017,  is  being  conducted  with 
developmental  test  aircraft  with  fuel  system  monitoring. 
The  program  has  developed  and  tested  a  correction 
involving  the  addition  of  a  relief  line  controlled  by  a  check 
valve  to  vent  the  affected  siphon  tanks  on  the  F-35B, 
which  has  very  similar  fuel  system  siphoning  architecture 
as  the  F-35C.  However,  the  program  has  not  tested  the 
pressure  relief  design  in  flight  on  an  F-35C.  Fleet  F-35C 
aircraft  are  limited  to  3.0  g’s  when  fully  fueled  and  the 
allowable  g  is  increased  as  fuel  is  consumed,  reaching  the 
full  Block  2B  envelope  of  6.0  g’s  at  roughly  43  percent 
of  total  fuel  quantity  remaining.  Until  relieved  of  the 
g  restrictions,  operational  units  will  have  to  adhere  to  a 
reduced  maneuvering  (i.e.,  less  “g  available”)  envelope  in 
operational  planning  and  tactics;  for  example,  managing 
threat  engagements  and  escape  maneuvers  when  in 
the  restricted  envelope  where  less  g  is  available.  This 
restriction  creates  an  operational  challenge  when  forward 
operating  locations  or  air  refueling  locations  are  close  to 
the  threat/target  arena. 

-  Air  refueling  with  strategic  tankers  (KC-135  and  KC-10) 
was  restricted  to  use  of  centerline  BDA  refueling  only. 
Refueling  from  tanker  wing  pods  was  prohibited  due  to 
response  anomalies  from  the  hose  and  reel  assemblies 
and  the  F-35C  aircraft  with  the  air  refueling  receptacle 
deployed. 

-  The  Patuxent  River  test  center  (Maryland)  conducted 
an  assessment  of  the  effects  of  transonic  roll  off  (TRO), 
which  is  an  un-commanded  roll  at  transonic  Mach 
numbers  and  elevated  angles  of  attack.  The  test  center 
also  assessed  buffet,  which  is  the  impact  of  airflow 
separating  from  the  leading  edge  of  the  wing  that  collides 
and  “buffets”  aft  areas  of  the  wing  and  aircraft  on  basic 
fighter  maneuvering.  TRO  and  buffet  occur  in  areas  of  the 
maneuvering  envelope  that  cannot  be  sustained  for  long 
periods  of  time,  as  energy  depletes  quickly  and  airspeed 
transitions  out  of  the  flight  region  where  these  conditions 
manifest.  However  fleeting,  these  areas  of  the  envelope 
are  used  for  critical  maneuvers.  The  testing  determined 
that  TRO,  observed  to  cause  up  to  8  degrees  angle  of  bank, 
adversely  affected  performance  in  defensive  maneuvering 
where  precise  control  of  bank  angles  and  altitude  must  be 
maintained  while  the  F-35C  is  in  a  defensive  position  and 
the  pilot  is  monitoring  an  offensive  aircraft.  The  test  pilots 
observed  less  of  an  effect  when  the  F-35C  is  conducting 
offensive  maneuvering.  However,  buffet  degrades  precise 
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aircraft  control  and  the  readability  of  heads-up-display 
symbology  in  the  HMDS  during  execution  of  certain 
critical  offensive  and  defensive  tasks,  such  as  defensive 
maneuvers. 

-  The  program  completed  two  test  flights  in  February  with 
CF-2,  an  instrumented  flight  sciences  test  aircraft  modified 
with  spoilers,  to  investigate  the  effects  on  flying  qualities 
when  using  control  laws  to  deploy  spoilers  in  the  flight 
regions  where  buffet  and  TRO  manifest  (between  Mach 
0.92  and  1.02  and  above  6  degrees  angle-of-attack). 

■  Testing  showed  the  spoilers  reduced  buffet  at  some  flight 
conditions,  but  also  may  increase  buffet  under  other 
flight  conditions,  and  reduced  the  magnitude  of  TRO 
when  experienced;  an  observation  predicted  by  wind 
tunnel  testing. 

■  Pilots  reported  that  spoilers  made  a  measurable 
difference  in  the  buffet-laden  region  of  the  flight 
envelope  but,  due  to  the  transient  nature  of  buffet,  the 
operational  significance  may  be  low. 

■  Operational  testing  of  the  F-35C  will  need  to  assess 
the  effect  of  TRO  and  buffet  on  overall  mission 
effectiveness. 

•  Weight  management  is  important  for  meeting  air  vehicle 
performance  requirements,  including  the  KPP  for  recovery 
approach  speed  to  the  aircraft  carrier,  and  structural  life 
expectations.  These  estimates  are  based  on  measured 
weights  of  components  and  subassemblies,  calculated 
weights  from  approved  design  drawings  released  for 
build,  and  estimated  weights  of  remaining  components. 

These  estimates  are  used  to  project  the  weight  of  the 

first  Lot  8  F-35C  aircraft  (CF-28)  planned  for  delivery  in 
March  2016,  which  will  be  the  basis  for  evaluating  contract 
specification  compliance  for  aircraft  weight. 

-  The  current  F-35C  estimate  of  34,582  pounds  is  286 
pounds  (less  than  1  percent)  below  the  planned  not-to- 
exceed  weight  of  34,868  pounds. 

-  The  program  will  need  to  ensure  the  actual  aircraft  weight 
meets  predictions  and  continue  rigorous  management 

of  the  actual  aircraft  weight  beyond  the  technical 
performance  measurements  of  contract  specifications  in 
CY16.  The  program  will  need  to  accomplish  this  through 
the  balance  of  SDD  to  avoid  performance  degradation  that 
would  affect  operational  capability. 

Mission  Systems 

Flight  Test  Activity  with  AF-3,  AF-6,  AF-7,  BF-4,  BF-5,  BF- 1 7,  BF- 1 8, 
CF-3,  and  CF-8  Flight  Test  Aircraft  and  Software  Development 
Progress 

•  Mission  systems  are  developed,  tested,  and  fielded  in 
incremental  blocks  of  capability. 

-  Block  1.  The  program  designated  Block  1  for  initial 
training  capability  in  two  increments:  Block  1A  for  Lot  2 
(12  aircraft)  and  Block  IB  for  Lot  3  aircraft  (17  aircraft). 
No  combat  capability  is  available  in  either  Block  1 
increment.  The  Services  have  upgraded  a  portion  of  these 


aircraft  to  the  Block  2B  configuration  through  a  series  of 
modifications  and  retrofits.  As  of  the  end  of  November, 

9  F-35A  and  12  F-35B  aircraft  had  been  modified  to  the 
Block  2B  configuration  and  4  F-35A  were  undergoing 
modifications.  Two  F-35B  aircraft,  which  are  on  loan  to 
the  Edwards  AFB  test  center  to  support  mission  systems 
developmental  flight  testing,  have  been  modified  to  the 
Block  3F  configuration,  leaving  one  F-35A  and  one  F-35B 
in  the  Block  IB  configuration.  Additional  modifications 
will  be  required  to  configure  these  aircraft  in  the  Block  3F 
configuration. 

-  Block  2A.  The  program  designated  Block  2A  for 
advanced  training  capability  and  delivered  aircraft  in 
production  Lots  4  and  5  in  this  configuration.  No  combat 
capability  is  available  in  Block  2A.  The  U.S.  Services 
accepted  62  aircraft  in  the  Block  2A  configuration 

(32  F-35A  aircraft  in  the  Air  Force,  19  F-35B  aircraft  in 
the  Marine  Corps,  and  11  F-35C  aircraft  in  the  Navy). 
Similar  to  the  Block  1A  and  Block  IB  aircraft,  the 
Services  have  upgraded  these  aircraft  to  the  Block  2B 
configuration  with  modifications  and  retrofits,  although 
fewer  modifications  were  required.  By  the  end  of 
September,  all  62  Lot  4  and  5  aircraft  had  been  modified  to 
the  Block  2B  configuration.  One  F-35C  aircraft,  which  is 
on  loan  to  the  Edwards  AFB  test  center,  has  been  modified 
to  the  Block  3F  configuration  to  support  mission  systems 
developmental  flight  testing.  Additional  modifications  will 
be  required  to  fully  configure  these  aircraft  in  the  Block  3F 
configuration. 

-  Block  2B.  The  program  designated  Block  2B  for  initial, 
limited  combat  capability  for  selected  internal  weapons 
(AIM-120C,  GBU-31/32  JDAM,  and  GBU-12).  This 
block  is  not  associated  with  the  delivery  of  any  lot  of 
production  aircraft.  Block  2B  mission  systems  software 
began  flight  testing  in  February  2013  and  finished  in 
April  2015.  Block  2B  is  the  software  that  the  Marine 
Corps  accepted  for  the  F-35B  Initial  Operational 
Capability  (IOC)  configuration. 

-  Block  3i.  The  program  designated  Block  3i  for  delivery 
of  aircraft  in  production  Lots  6  through  8,  as  these  aircraft 
include  a  set  of  upgraded  integrated  core  processors 
(referred  to  as  Technical  Refresh  2,  or  TR-2).  The 
program  delivered  Lot  6  aircraft  with  a  Block  3i  version 
that  included  capabilities  equivalent  to  Block  2A  in 

Lot  5.  Lot  7  aircraft  are  being  delivered  with  capabilities 
equivalent  to  Block  2B,  as  will  Lot  8  aircraft.  Block  3i 
software  began  flight  testing  in  May  2014  and  completed 
baseline  testing  in  October  2015,  eight  months  later  than 
planned  in  the  Integrated  Master  Schedule  (IMS).  The 
program  completed  delivery  of  the  U.S.  Service’s  Lot  6 
aircraft  in  2015  (18  F-35A,  6  F-35B,  and  7  F-35C  aircraft). 
The  delivery  of  Lot  7  aircraft  began  in  August  2015,  with 
four  F-35A  aircraft  delivered  to  the  U.S.  Air  Force.  By 
the  end  of  November,  the  program  had  delivered  13  F-35A 
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Lot  7  aircraft  to  the  U.S.  Air  Force  and  two  F-35B  Lot  7 
aircraft  to  the  Marine  Corps. 

-  Block  3F.  The  program  designated  Block  3F  as  the 
full  SDD  capability  for  production  Lot  9  and  later. 

Flight  testing  with  Block  3F  software  on  the  F-35  test 
aircraft  began  in  March  2015.  Aircraft  from  production 
Lots  2  through  5  will  need  to  be  modified,  including 
the  installation  of  TR-2  processors,  to  have  Block  3F 
capabilities. 

•  Mission  systems  testing  focused  on: 

-  Completing  Block  2B  flight  testing 

-  Completing  Block  3i  flight  testing 

-  Beginning  Block  3F  flight  testing 

-  Regression  testing  of  corrections  to  deficiencies  identified 
in  Block  2B  and  Block  3i  flight  testing 

-  Testing  of  the  Gen  III  HMDS 

•  The  six  mission  systems  developmental  flight  test  aircraft 
assigned  to  the  Edwards  AFB  test  center  flew  an  average 
rate  of  6.8  flights  per  aircraft,  per  month  in  CY15  through 
November,  exceeding  the  planned  rate  of  6.0  by  13  percent, 
and  flew  107  percent  of  the  planned  flights  (483  flights 
accomplished  versus  452  planned). 

•  The  program  prioritized  flight  test  activity  early  in  the  year 
to  complete  Block  2B  flight  testing.  The  program  declared 
testing  complete  on  Block  2B  software  at  the  end  of  April. 
The  program  made  the  decision,  in  part,  based  on  schedule, 
to  support  the  need  for  moving  forward  with  Block  3i  and 
Block  3F  testing,  which  required  modifying  the  mission 
systems  test  aircraft  with  upgraded  TR-2  processors. 

•  The  Edwards  AFB  test  center  used  production  operational 
test  aircraft,  assigned  to  the  operational  test  squadron 
there,  to  assist  in  accomplishing  developmental  test  points 
of  Block  2B  capabilities  throughout  the  year,  including 
augmenting  testing  requiring  formation  flight  operations. 

Mission  Systems  Assessment 

•  Block  2B  Development 

-  The  program  completed  Block  2B  mission  systems  testing 
and  provided  a  fleet  release  version  of  the  software  with 
deficiencies  identified  during  testing. 

-  The  program  attempted  to  correct  deficiencies  in  the 
fusion  of  information — from  the  sensors  on  a  single 
aircraft  and  between  aircraft  in  formation — identified 
during  flight  testing  in  late  CY14  and  early  CY15  of  the 
planned  final  Block  2B  software  version.  The  test  team 
flew  an  “engineering  test  build”  (ETB)  of  the  software 
designated  2BS5.2ETB.  on  17  test  flights  using  3  different 
mission  systems  test  aircraft  in  March.  Although  some 
improvement  in  performance  was  observed,  distinguishing 
ground  targets  from  clutter  continued  to  be  problematic. 

As  a  residt,  the  program  chose  to  field  the  final  (prior 

to  the  ETB)  version  of  Block  2B  software  and  defer 
corrections  to  Block  3i  and  Block  3F. 

-  Five  mission  systems  deficiencies  were  identified  by  the 
Air  Force  as  “must  fix”  for  the  final  Block  3i  software 
release,  while  the  Marine  Corps  did  not  require  the 
deficiencies  to  be  fixed  in  Block  2B.  These  deficiencies 
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were  associated  with  information  displayed  to  the  pilot 
in  the  cockpit  concerning  performance  and  accuracy  of 
mission  systems  functions  related  to  weapon  targeting, 
radar  tracking,  status  of  fused  battlespace  awareness  data, 
health  of  the  integrated  core  processors,  and  health  of  the 
radar.  Another  deficiency  was  associated  with  the  time 
it  takes  to  download  files  in  order  to  conduct  a  mission 
assessment  and  debriefing. 

-  Continuing  to  work  the  Block  2B  deficiencies  would 
have  delayed  the  necessary  conversion  of  the  labs  and  the 
developmental  test  aircraft  to  the  Block  3i  and  Block  3F 
configuration,  delaying  the  ability  for  the  program  to 
complete  Block  3i  testing  needed  for  delivery  of  aircraft 
from  production  Lots  6  and  7,  and  starting  flight  testing  of 
Block  3F  software. 

-  The  program  deferred  two  WDA  events  from  Block  2B 
to  Block  3F  as  a  result  of  the  decision  to  stop  Block  2B 
testing  in  April.  This  deferred  work  will  add  more 
pressure  to  the  already  demanding  schedule  of  Block  3F 
WDA  events. 

-  The  program  attempted  to  correct  known  deficiencies  from 
flight  testing  of  Block  2B  software  in  the  Block  3i  software 
product  line  (i.e.,  mission  systems  labs  and  Block  3i 
flight  test  aircraft).  The  program  corrected  some  of  these 
deficiencies  and,  as  of  the  end  of  November  2015,  planned 
to  transfer  these  corrections  to  a  new  version  of  Block  2B 
software  (2BS5.3)  for  a  release  in  CY16.  In  order  to 
accomplish  this,  the  program  needs  to  use  aircraft  from  the 
operational  test  fleet,  which  will  still  be  in  the  Block  2B 
configuration,  to  test  the  2BS5.3  software.  However,  this 
entire  process  introduces  inefficiencies  in  the  program’s 
progress  for  developing  and  testing  Block  3F  software. 

•  Block  2B  Fleet  Release 

-  The  program  finished  Block  2B  developmental  testing 
in  May  (mission  systems  testing  completed  in  April,  and 
F-35B  flight  sciences  testing  completed  in  May)  and 
provided  the  necessary  data  for  the  Service  airworthiness 
authorities  to  release  Block  2B  capabilities  to  their 
respective  fleets.  The  Marine  Corps  released  Block 

2B  to  the  F-35B  fielded  units  in  June,  the  Air  Force  to 
the  F-35  A  units  in  August,  and  the  Navy  to  the  F-35C 
units  in  October.  The  fleet  release  enabled  the  Services 
to  load  Block  2B  software  on  their  aircraft,  provided 
they  had  been  modified  at  least  in  part  to  the  Block  2B 
configuration. 

-  Because  of  the  limited  combat  capability  provided  in 
Block  2B,  if  the  Block  2B  F-35  aircraft  will  be  used  in 
combat,  it  will  need  the  support  of  a  command  and  control 
system  that  will  assist  in  target  acquisition  and  to  control 
weapons  employment  for  the  limited  weapons  carriage 
available.  If  in  an  opposed  combat  scenario,  the  F-35 
Block  2B  aircraft  would  need  to  avoid  threat  engagement 
and  would  require  augmentation  by  other  friendly  forces. 
The  Block  2B  fleet  release  carries  maneuver  and  envelope 
restrictions  that,  although  agreed  to  by  the  Services  during 
requirements  reviews,  will  also  limit  effectiveness: 
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■  For  the  F-35A,  the  airspeed  at  which  the  weapons  bay 
doors  can  be  open  in  flight  (550  knots  or  1.2  Mach) 

is  less  than  the  maximum  aircraft  speed  allowable 
(700  knots  or  1.6  Mach).  Such  a  restriction  will  limit 
tactics  to  employment  of  weapons  at  lower  speeds  and 
may  create  advantages  for  threat  aircraft  being  pursued 
by  the  F-35A. 

■  For  the  F-35A,  the  airspeed  at  which  countermeasures 
can  be  used  is  also  less  than  the  maximum  speed 
allowable,  again  restricting  tactical  options  in  scenarios 
where  F-35A  pilots  are  conducting  defensive  maneuvers 

-  The  program  formally  vets  deficiency  reports  submitted 
by  test  and  operational  organizations.  The  formal  process 
assigns  deficiency  reports  to  categories  correlating  to 
urgency  for  correction.  Category  I  deficiencies  are 
those  which  may  cause  death,  severe  injury,  or  severe 
occupational  illness;  may  cause  loss  or  major  damage 

to  a  weapon  system;  critically  restrict  the  combat 
readiness  capabilities  of  the  using  organization;  or  result 
in  a  production  line  stoppage.  Category  II  deficiencies 
are  those  that  impede  or  constrain  successful  mission 
accomplishment  (but  do  not  meet  the  safety  or  mission 
impact  criteria  of  a  Category  I  deficiency).  As  of  the 
end  of  October  2015,  91  Category  1  (mission  or  safety 
of  flight  impact,  27)  and  Category  2  (mission  impact, 

64)  high-severity  deficiencies  in  the  Hill  Block  2B 
configuration  (air  vehicle,  propulsion,  mission  systems) 
were  not  yet  resolved  by  the  program.  Of  these  91,  43  are 
assigned  to  mission  systems  engineering  for  resolution. 

-  In  addition  to  the  mission  systems  deficiencies  cited  above, 
the  Block  2B  fleet  aircraft  are  restricted  by  fuel  system 
deficiencies: 

■  All  variants  of  the  fleet  Block  2B  aircraft  are  restricted 
from  exceeding  3  gs  in  symmetric  maneuvers  when 
fully  fueled  in  order  to  avoid  exceeding  the  allowable 
pressure  in  the  siphon  fuel  tanks.  The  allowable 

g  increases  as  fuel  is  consumed.  The  program  has 
developed  and  tested  a  hardware  correction  to  the 
problem  for  the  F-35B;  corrections  for  the  F-35A  and 
F-35C  are  still  in  work.  Modification  kits  for  installation 
on  fielded  production  aircraft  are  currently  in  production 
for  the  F-35B  and  aircraft  delivered  in  production  Lot 
8  will  include  the  correct  hardware.  This  modification 
will  restore  the  envelope  of  the  F-35B. 

■  The  program  lifted  the  restriction  preventing  the 
F-35B  from  flying  within  25  nautical  miles  of  known 
lightning  prior  to  the  declaration  of  IOC;  however,  the 
program  has  added  a  restriction  from  taxiing  or  taking 
off  within  25  nautical  miles  of  known  lightning  because 
of  only  a  partial  software  mitigation  to  the  siphon  tank 
overpressure  problem.  The  program  plans  to  field  a 
new  software  release  in  1QCY16,  which  will  enable  a 
hardware  correction  to  the  overpressure  problem,  once 
fielded  F-35B  aircraft  are  retrofitted  with  the  hardware 
modification. 


Block  3i 

-  Block  3i  flight  testing  began  in  May  2014  with  version 
3iRl,  derived  from  Block  2A  software,  six  months  later 
than  planned  in  the  IMS.  The  latest  version  of  Block  3i 
software — 3iR6 — began  flight  testing  in  July  2015  and 
was  derived  from  the  latest  version  of  Block  2B  software. 
Block  3i  mission  systems  flight  testing  completed  in 
October  2015,  eight  months  later  than  planned  in  the  IMS. 

-  Since  the  program  planned  to  not  introduce  new 
capabilities  in  Block  3i,  the  test  plan  was  written  to 
confirm  Block  3i  had  equivalent  capabilities  to  those 
demonstrated  in  Block  2A  (for  3iRl)  and  Block  2B  (for 
subsequent  versions  of  Block  3i  software).  The  program’s 
plan  required  completion  of  5 14  baseline  test  points 

by  mid-February  2015,  with  additional  development, 
regression,  and  discovery  points  flown  as  necessary  for 
each  increment  of  software  to  address  deficiencies.  The 
program  completed  Block  3i  mission  systems  testing  by 
accomplishing  469  of  the  514  baseline  Block  3i  test  points, 
or  91  percent.  Of  the  45  test  points  remaining,  6  were 
transferred  for  completion  in  Block  3F  and  the  remaining 
39  were  designated  as  “no  longer  required.”  The  program 
executed  an  additional  515  test  points.  Of  those  515 
points,  151  were  allocated  in  the  budgeted  non-baseline 
points  for  the  year,  and  the  364  additional  points  represent 
growth  in  Block  3i  testing.  These  364  additional  points, 
needed  to  accomplish  the  469  baseline  test  points, 
represent  a  growth  of  78  percent,  which  is  much  higher 
than  the  non-baseline  budgeted  of  30  percent  planned  by 
the  program  to  complete  Block  3i  testing. 

-  Results  from  3iR6  flight  testing  demonstrated  partial 
fixes  to  the  five  “must  fix  for  Air  Force  IOC”  deficiencies, 
showing  some  improved  performance.  Poor  stability 

in  the  radar,  however,  required  multiple  ground  and 
flight  restarts,  a  condition  that  will  reduce  operational 
effectiveness  in  combat. 

-  Instabilities  discovered  in  the  Block  3i  configuration 
slowed  progress  in  testing  and  forced  development  of 
additional  software  versions  to  improve  performance.  Two 
additional  versions  of  the  3iR5  software  were  created  in 
an  attempt  to  address  stability  in  start-up  of  the  mission 
systems  and  inflight  stability  of  the  radar.  Overall, 

radar  performance  has  been  less  stable  in  the  Block  3i 
configuration  than  in  Block  2B.  The  test  centers  developed 
a  separate  “radar  stability”  series  of  tests — including  both 
ground  startup  and  inflight  testing — to  characterize  the 
stability  problems.  Radar  stability  is  measured  in  terms 
of  the  number  of  times  per  flight  hour  that  either  of  these 
events  occurred:  a  failure  event  requiring  action  by  the 
pilot  to  reset  the  system;  or,  a  stability  event  where  the 
system  developed  a  fault,  which  affected  performance, 
but  self-corrected  without  pilot  intervention.  For  the  last 
version  of  Block  2B  software — 2BS5.2 — the  test  team 
measured  a  mean  time  between  stability  or  failure  event 
of  32.5  hours  over  nearly  200  hours  of  flight  testing.  For 
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3iR6,  the  time  interval  between  events  was  4.3  hours  over 
215  hours  of  flight  testing.  This  poor  radar  stability  will 
degrade  operational  mission  effectiveness  in  nearly  all 
mission  areas. 

-  Since  no  capabilities  were  added  to  Block  3i,  only  limited 
corrections  to  deficiencies,  the  combat  capability  of  the 
initial  operational  Block  3i  units  will  not  be  noticeably 
different  than  the  Block  2B  units.  If  the  Block  3i  F-35 
aircraft  will  be  used  in  combat,  they  will  need  equivalent 
support  as  for  the  Block  2B  F-35  aircraft,  as  identified 
previously  in  this  report. 

-  As  of  the  end  of  October,  a  total  of  nine  Category  1  (three 
mission  or  safety  of  flight  impact)  and  Category  2  (six 
mission  impact)  high-severity  deficiencies  in  the  full 
Block  3i  configuration  (air  vehicle,  propulsion,  mission 
systems)  were  unresolved.  Eight  of  these  nine  are 
assigned  to  mission  systems  engineering  for  resolution. 

-  Based  on  these  Block  3i  performance  issues,  the  Air  Force 
briefed  that  Block  3i  mission  capability  is  at  risk  of  not 
meeting  IOC  criteria  to  the  Joint  Requirements  Oversight 
Council  (JROC)  in  December  2015.  The  Air  Force 
recently  received  its  first  Block  3i  operational  aircraft  and 
is  assessing  the  extent  to  which  Block  3i  will  meet  Air 
Force  IOC  requirements;  this  assessment  will  continue  into 
mid-2016. 

Block  3F 

-  Block  3F  flight  testing  began  in  March  2015,  six 
months  later  than  the  date  planned  by  the  program  after 
restructuring  in  2012. 

-  As  of  the  end  of  November,  a  total  of  674  Block  3F 
baseline  test  points  had  been  completed,  compared 
to  575  planned  (17  percent  more  than  planned).  An 
additional  653  development  and  regression  points  were 
flown,  all  of  which  were  part  of  the  budgeted  non-baseline 
points  for  the  year. 

-  Since  many  of  the  baseline  test  points — which  are  used  to 
confirm  capability — cannot  be  tested  until  later  versions 
of  the  Block  3F  software  are  delivered  in  CY16  and 
CY17,  the  program  allocated  a  large  number  of  test  points 
(979  for  CY15)  for  development  and  regression  of  the 
software,  while  expecting  to  accomplish  only  677  baseline 
test  points  in  CY15.  The  total  planned  amount  of  baseline 
test  points  to  complete  Block  3F  are  approximately  5,467; 
combined  with  the  planned  non-baseline  test  points  in 

the  approved  test  plan,  there  are  approximately  7,230  test 
points  for  Block  3F. 

-  Due  to  the  later-than-planned  start  of  Block  3F  mission 
systems  testing  (6  months  late),  the  large  amount  of 
planned  baseline  test  points  remaining  (88  percent),  and 
the  likelihood  of  the  need  for  additional  test  points  to 
address  discoveries  and  fixes  for  deficiencies,  the  program 
will  not  be  able  to  complete  Block  3F  missions  systems 
flight  test  by  the  end  of  October  2016,  as  indicated  by  the 
IMS.  Instead,  the  program  will  likely  not  finish  Block  3F 
development  and  flight  testing  prior  to  January  2018, 
based  on  the  following: 


■  Continuing  a  six  test  point  per  flight  accomplishment 
rate,  which  is  equal  to  the  CY15  rate  observed  through 
the  end  of  November 

■  Continuing  a  flight  rate  of  6.8  flights  per  month,  as  was 
achieved  through  the  end  of  November  2015,  exceeding 
the  planned  rate  of  6  flights  per  month  (if  the  flight  rate 
deteriorates  to  the  planned  rate  of  6  flights  per  month, 
then  testing  will  not  complete  until  May  2018). 

■  Completing  the  full  Block  3F  test  plan  (i.e.,  all  original 
7,230  baseline  and  budgeted  non-baseline  test  points  in 
the  Block  3F  joint  test  plan) 

■  Continuing  the  CY15  discovery  rate  of  5  percent 

-  The  program  currently  tracks  337  total  Category  1 
(42  mission  or  safety  of  flight  impact)  and  Category  2 
(295  mission  impact)  high-severity  deficiencies  in  the  full 
Block  3F  configuration  (air  vehicle,  propulsion,  mission 
systems),  of  which  200  are  assigned  to  the  mission 
systems  engineering  area  for  resolution.  An  additional 
100  Category  1  and  Category  2  high-severity  deficiencies 
are  unresolved  from  Block  2B  and  Block  3i  configurations, 
of  which  5 1  are  assigned  to  mission  systems  for  resolution. 
It  remains  to  be  determined  how  many  of  these  the 
program  will  be  able  to  correct  in  later  Block  3F  versions. 
If  any  of  these  deficiencies  are  not  resolved  in  the  planned 
Block  3F  design,  additional  efforts  to  isolate  causes,  and 
design  and  verify  fixes  will  increase  the  amount  of  time 
needed  to  complete  Block  3F  development  and  testing. 

-  The  program  could,  as  has  been  the  case  in  testing 
previous  software  increments,  determine  test  points  in  the 
plan  are  no  longer  required  for  the  Block  3F  fleet  release. 
However,  the  program  will  need  to  ensure  that  deleting 
and/or  deferring  testing  from  Block  3F  before  the  end  of 
SDD  and  start  of  IOT&E  does  not  increase  the  likelihood 
of  discovery  in  IOT&E  or  affect  the  evaluation  of  mission 
capability.  Whatever  capability  the  program  determines  as 
ready  for  IOT&E  will  need  to  undergo  the  same  rigorous 
and  realistic  combat  mission-focused  testing  as  a  fully 
functioning  system. 

-  Block  3F  mission  systems  capabilities  require  more 
complex  test  scenarios  than  prior  versions  of  mission 
systems.  It  requires  testing  involving  significantly  more 
complex  threat  behavior  and  threat  densities  on  the  test 
ranges  than  was  used  in  prior  versions  of  mission  systems. 
Additionally,  Block  3F  capability  requires  more  testing  in 
multi-ship  formations. 

Helmet  Mounted  Display  System  (HMDS) 

•  The  HMDS  is  pilot  flight  equipment.  It  has  a  display  on  the 
visor  that  provides  the  primary  visual  interface  between  the 
pilot  and  the  air  vehicle  and  mission  systems.  The  HMDS 
was  envisioned  to  replace  a  traditional  cockpit-mounted 
“heads-up  display”  and  night  vision  goggles.  It  projects 
imagery  from  sensors  onto  the  helmet  visor,  which  is 
intended  to  enhance  pilot  situational  awareness  and  reduce 
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workload.  In  2010,  the  Program  Office  identified  significant 
deficiencies  and  technical  risk  in  the  HMDS. 

•  The  program  created  a  “dual-path”  approach  to  recover 
required  capability. 

-  One  path  was  to  fix  the  existing  Generation  II  (Gen  II) 
HMDS  through  redesign  of  the  night  vision  system/camera 
and  electro-optical/infrared  sensor  imagery  integration  on 
the  visor. 

-  The  second  path  was  to  switch  to  an  alternate  helmet 
design  incorporating  legacy  night  vision  goggles  and 
projecting  sensor  imagery  only  on  cockpit  displays. 

-  The  program  terminated  the  dual  path  approach  in  2013 
and  decided  to  move  forward  with  fixes  to  the  existing 
Gen  II  HMDS  which  created  the  Gen  III  HMDS 

•  The  Gen  II  HMDS  was  fielded  with  Block  2  and  earlier 
configurations  of  aircraft.  The  program  developed  and 
tested  improvements  to  address  deficiencies  in  stability  of 
the  display  (referred  to  as  “jitter”),  latency  in  the  projection 
of  Distributed  Aperture  System  (DAS)  imagery,  and  light 
leakage  onto  the  display  under  low-light  conditions  (referred 
to  as  “green  glow”).  However,  adequate  improvements  to 
the  night  vision  camera  acuity  were  not  completed  and  pilots 
were  prohibited  from  using  the  night  vision  camera.  Pilot 
use  of  the  DAS  imagery  was  also  restricted. 

•  The  Gen  III  HMDS  is  intended  to  resolve  all  of  the  above 
deficiencies.  It  is  a  requirement  for  Air  Force  IOC  in  2016, 
and  will  be  used  to  complete  SDD  and  IOT&E  in  2018.  The 
following  provide  Gen  III  HMDS  details: 

-  It  includes  a  new  higher-resolution  night  vision  camera, 
software  improvements,  faster  processing,  and  changes  to 
the  imagery  projection  systems  for  the  visor. 

-  It  requires  aircraft  with  Block  3i  hardware  and  software. 

-  Developmental  flight  testing  began  in  December  2014 
and  will  continue  into  2016  with  primary  flight  reference 
testing. 

-  Operational  testing  will  occur  in  tests  conducted  to  support 
the  Air  Force  IOC  in  2016  (Block  3i),  and  in  IOT&E 
(Block  3F). 

-  It  will  be  used  with  all  Lot  7  aircraft,  which  are  being 
delivered  now,  and  later  deliveries. 

-  Later-than-planned  escape  system  qualification  delayed 
Gen  III  HMDS  deliveries  to  the  field;  the  program  plans 
full  flight  clearance  to  occur  in  2016. 

•  Results  of  the  Gen  III  HMDS  performance  during 
developmental  testing  thus  far  indicate  the  following: 

-  Symbology  jitter  and  alignment.  Some  corrections  were 
made  for  jitter  and  alignment  in  the  latest  configuration  of 
the  fielded  Gen  II  HMDS  via  modifications  to  the  display 
management  computer.  These  are  carried  into  the  Gen  III 
design.  Developmental  test  pilots  report  less  jitter  and 
proper  alignment.  However,  jitter  still  occurs  in  regimes 
of  high  buffet  (i.e.,  during  high  g  or  high  angle  of  attack 
maneuvering).  Operational  testing  in  heavy  maneuvering 
environments  is  needed  to  determine  if  further  attention 
will  be  required. 


-  Green  glow  (difficulty  setting  symbology  intensity  level 
without  creating  a  bright  green  glow  around  perimeter  of 
display).  The  Gen  III  HMDS  includes  new  displays  with 
higher  contrast  control,  which  has  reduced  green  glow 
compared  to  Gen  II;  the  phenomena  still  exists,  but  at  a 
manageable  level,  according  to  developmental  test  pilots. 
Developmental  test  pilots  were  able  to  air  refuel  and 
operate  in  “no  moon”  low  illumination  conditions  at  night. 
Simulated  carrier  approaches  were  also  conducted  at  San 
Clemente  Island  off  the  coast  of  California  and  during 
carrier  trials  in  October  2015.  Operational  testing  in  high 
mission  task  loads  is  also  needed  to  confirm  if  further 
adjustments  are  needed. 

-  Latency  (projected  imagery  lagging  head 
movement/placement).  The  Gen  III  HMDS  includes 
faster  processing  to  reduce  latency  in  night  vision  camera 
imagery  and  DAS  imagery  projected  onto  the  visor.  The 
update  rate  in  the  Gen  III  HMDS  is  twice  that  of  the 
Gen  II.  Developmental  test  pilots  reported  improvement 
in  this  area.  Nonetheless,  pilots  have  to  “learn”  an 
acceptable  head-movement  rate;  that  is,  they  cannot  move 
their  heads  too  rapidly.  However,  operational  testing  in 
these  environments  is  needed  to  determine  if  the  problem 
is  resolved  and  pilot  workload  is  reduced,  especially 
during  weapons  employment. 

-  Night  vision  camera  resolution.  The  Gen  II  camera 
included  a  single  1280  x  1024  pixel  night  vision  sensor. 
The  Gen  111  camera  includes  two  1600  x  1200  sensors  and 
additional  image  processing  software  changes,  which  are 
intended  to  provide  improved  resolution  and  sensitivity. 
Developmental  test  pilots  reported  better  acuity  allowing 
pilots  to  accomplish  mission  tasks.  Operational  testing 
under  high  mission  task  loads  will  determine  if  further 
improvement  is  needed. 

Mission  Data  Load  Development  and  Testing 

•  F-35  effectiveness  in  combat  relies  on  mission  data 
loads — which  are  a  compilation  of  the  mission  data  files 
needed  for  operation  of  the  sensors  and  other  mission 
systems — working  in  conjunction  with  the  system  software 
data  load  to  drive  sensor  search  parameters  so  that  the  F-35 
can  identify  and  correlate  sensor  detections,  such  as  threat 
and  friendly  radar  signals.  The  contractor  team  produced 
an  initial  set  of  files  for  developmental  testing  during 
SDD,  but  the  operational  mission  data  loads — one  for  each 
potential  major  geographic  area  of  operation — are  being 
developed,  tested,  and  produced  by  a  U.S.  government  lab, 
the  U.S.  Reprogramming  Lab  (USRL),  located  at  Eglin 
AFB,  Florida,  which  is  operated  by  government  personnel 
from  the  Services.  The  Air  Force  is  the  lead  Service.  These 
mission  data  loads  will  be  used  for  operational  testing  and 
fielded  aircraft,  including  the  Marine  Corps  and  Air  Force 
IOC  aircraft.  The  testing  of  the  USRL  mission  data  loads  is 
an  operational  test  activity,  as  was  arranged  by  the  Program 
Office  after  the  restructure  that  occurred  in  2010. 
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•  Significant  deficiencies  exist  in  the  USRL  that  preclude 

efficient  development  of  effective  mission  data  loads.  Unless 
remedied,  these  deficiencies  will  cause  significant  limitations 
for  the  F-35  in  combat  against  existing  threats.  These 
deficiencies  apply  to  multiple  potential  theaters  of  operation 
and  affect  all  variants  and  all  Services. 

-  In  February  2012,  DOT&E  recommended  upgrades  to  the 
USRL  to  overcome  the  significant  shortfalls  in  the  ability 
of  the  lab  to  provide  a  realistic  environment  for  mission 
data  load  development  and  testing.  The  Department 
provided  a  total  of  $45  Million  in  resources  to  overcome 
these  shortfalls,  with  the  funding  beginning  in  2013. 
Unfortunately,  due  to  the  Program  Office  leadership’s 
failure  to  accord  the  appropriate  priority  to  implementing 
the  required  corrections,  not  until  last  year  did  the  program 
move  to  investigate  the  deficiencies  in  the  lab  and  build 

a  plan  for  corrections,  and  only  recently  did  it  initiate 
the  process  of  contracting  for  improvements,  which  has 
yet  to  finalize  at  the  time  of  this  report.  The  status  of  the 
Department’s  investment  is  not  clear. 

-  The  program’s  belated  2014  investigation  confirmed 
the  nature  and  severity  of  the  shortfalls  that  DOT&E 
identified  in  2012.  The  analysis  also  identified  many 
other  gaps,  some  of  which  are  even  more  urgent  and 
severe  than  those  uncovered  by  DOT&E  three  years 
prior.  Failure  to  aggressively  address  the  deficiencies 
results  in  uncertainties  in  the  aircraft’s  capabilities  to  deal 
with  existing  threats;  uncertainties  that  will  persist  until 
the  deficiencies  have  been  overcome  and  which  could 
preclude  the  aircraft  from  being  operationally  effective 
against  the  challenging  threats  it  is  specifically  being 
fielded  to  counter.  The  program  planned  to  complete 
upgrades  to  the  lab  in  late  2017,  which  will  be  late  to  need 
if  the  lab  is  to  provide  a  mission  data  load  for  Block  3F 
tactics  development  and  preparation  for  IOT&E.  It  is 
important  to  note  that  many  of  these  deficiencies  apply 
equally  to  the  contractor’s  mission  systems  development 
labs  because  the  government  lab  is  essentially  a  copy  of 
one  of  the  mission  system  software  integration  test  labs  at 
the  contractor  facility. 

-  The  findings  of  the  program’s  2014  investigation  include: 

■  Shortfalls  in  the  ability  to  replicate  signals  of  advanced 
threats  with  adequate  fidelity  and  in  adequate  numbers 

■  Inability  to  adequately  and  coherently  stimulate  all 
signal  receivers  in  F-35  mission  systems 

■  Receiver  scan  scheduling  tools  do  not  function  correctly 
when  replicating  complex  threats 

■  Mission  data  file  generation  tools  errantly  combine 
emitter  modes 

■  Important  emitter  data  are  ignored  by  the  tools,  which 
adversely  affect  the  quality  of  the  mission  data  files 

■  Inability  to  edit  existing  mission  data  files,  a  condition 
which  requires  inefficient  processes  to  make  changes 
where  the  lab  technicians  must  reconstruct  the  entire 
mission  data  file  set  with  new/corrected  information 


-  The  program  must  make  these  modifications  before  the 
USRL  is  required  to  provide  the  Block  3F  mission  data 
load  for  tactics  development  and  preparations  for  IOT&E. 
The  program’s  2014  study,  while  agreeing  with  DOT&E 
that  significant  hardware  upgrades  are  needed,  has  not 
resulted  in  a  plan  to  procure  those  upgrades  in  time  for 
Block  3F  mission  data  load  development  and  verification. 
Despite  the  $45  Million  budget,  the  program  has  still 

not  designed,  contracted  for,  and  ordered  the  required 
equipment — a  process  that  will  take  at  least  two  years, 
not  counting  installation  and  check-out.  In  addition, 
despite  the  conclusions  of  the  2014  study  by  the  Program 
Office,  the  program  has  sub-optimized  the  upgrades  it  will 
eventually  put  on  contract  due  to  budgetary  constraints. 
Procuring  only  a  limited  number  of  signal  generators 
would  leave  the  USRL  with  less  capability  than  the  F-35 
Foreign  Military  Sales  Reprogramming  Lab.  This  decision 
constitutes  a  critical  error  on  the  part  of  the  program’s 
leadership. 

-  An  investment  greater  than  the  $45  Million  recommended 
by  DOT&E  in  2012  is  needed  to  address  all  necessary 
hardware  and  software  corrections  to  the  lab.  Although 
over  three  years  have  already  been  lost  to  inaction, 

the  Program  Office  still  does  not  plan  to  put  Block  3F 
upgrades  to  the  USRL  on  contract  until  late  in  2016. 

The  program  recently  briefed  that  once  the  equipment  is 
finally  ordered  in  2016,  it  would  take  at  least  two  years 
for  delivery,  installation,  and  check-out — after  IOT&E 
begins  (according  to  the  current  schedule  of  the  program 
of  record).  This  results  in  a  high  risk  to  both  a  successful 
IOT&E  and  readiness  for  combat.  When  deficiencies 
were  first  identified  in  2012,  there  was  time  to  make  early 
corrections  and  avoid,  or  at  least  significantly  reduce, 
the  risk  that  is  now  at  hand.  Instead,  due  to  the  failure  of 
leadership,  the  opposite  has  occurred. 

•  The  USRL  staff  submitted  a  plan  in  2013  for  the  operational 
testing  of  the  Block  2B  mission  data  loads,  which  was 
amended  by  the  test  team  per  DOT&E  instructions,  and 
approved  by  DOT&E.  The  plan  includes  multi-phased  lab 
testing  followed  by  a  series  of  flight  tests  before  release  to 
operational  aircraft. 

•  Because  the  program  elected  to  delay  the  arrival  of  the  USRL 
equipment  several  years,  a  significant  amount  of  schedule 
pressure  on  the  development  and  testing  of  the  Block  2B 
mission  data  loads  developed  in  2015.  The  USRL  staff  was 
required  to  truncate  the  planned  testing,  forgoing  important 
steps  in  mission  data  load  development,  optimization,  and 
verification,  and  instead,  apply  its  resources  and  manpower 
to  providing  a  limited  mission  data  load  in  June  2015  for 

the  Marine  Corps  IOC.  The  limited  extent  of  lab  and  flight 
testing  that  occurred  creates  uncertainties  in  F-35  combat 
effectiveness  that  must  be  taken  into  consideration  by 
fielded  operational  units  until  the  lab  is  able  to  complete 
optimization  and  testing  of  a  Block  2B  mission  data  load  in 
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accordance  with  the  plan.  This  additional  work  is  planned  to 
occur  in  early  2016. 

•  A  similar  sequence  of  events  may  occur  with  the  Air  Force 
IOC,  planned  for  August  2016  with  Block  3i.  Mission  data 
loads  must  be  developed  to  interface  with  the  system  data 
load,  and  they  are  not  forwards  or  backwards  compatible. 
Block  3i  mission  data  load  development  and  testing  will 
occur  concurrently  with  completion  of  Block  2B  mission 
data  loads,  creating  pressure  in  the  schedule  as  the  lab 
configuration  will  have  to  be  changed  to  accommodate  the 
development  and  testing  of  both  blocks. 

Weapons  Integration 

Block  2B 

•  The  program  terminated  Block  2B  developmental  testing 
for  weapons  integration  in  December  2015  after  completing 
12  of  the  15  planned  WDA  events.  The  program  had  planned 
to  complete  all  15  WDA  events  by  the  end  of  October  2014, 
but  delays  in  implementing  software  fixes  for  deficient 
performance  of  the  Electro-Optical  Targeting  System 
(EOTS),  radar,  fusion,  Multi-function  Advanced  Data  Link 
(MADL),  Link  16  datalink,  and  electronic  warfare  mission 
systems  slowed  progress. 

-  All  three  of  the  deferred  events  are  AIM- 120  missile  shot 
scenarios.  The  program  deferred  one  of  the  remaining 
events  to  Block  3i,  awaiting  mission  systems  updates  for 
radar  deficiencies.  The  program  completed  that  missile 
shot  scenario  in  September  2015  with  Block  3i  software. 
The  program  deferred  the  other  two  events  to  Block  3F 
due  to  mission  systems  radar,  fusion,  and  electronic 
warfare  system  deficiencies.  Fixes  to  Block  3F  capability 
are  needed  in  order  to  execute  these  scenarios. 

-  Eleven  of  the  12  completed  events  required  developmental 
test  control  team  intervention  to  overcome  system 
deficiencies  to  ensure  a  successful  event  (acquire  and 
identify  target,  engage  with  weapon).  The  program  altered 
the  event  scenarios  to  make  them  less  challenging  for 
three  of  these,  as  well  as  the  twelfth  event,  specifically 

to  work  around  F-35  system  deficiencies  (e.g.,  changing 
target  spacing  or  restricting  target  maneuvers  and 
countermeasures).  The  performance  of  the  Block  2B 
configured  F-35  in  combat  will  depend  in  part  on  the 
degree  to  which  the  enemy  conforms  to  these  narrow 
scenarios,  which  is  unlikely,  and  enables  the  success 
of  the  workarounds  necessary  for  successful  weapons 
engagement. 

•  Mission  systems  developmental  testing  of  system 
components  required  neither  operation  nor  full  functionality 
of  subsystems  that  were  not  a  part  of  the  component  under 
test.  The  developmental  test  teams  designed  the  individual 
component  tests  only  to  verify  compliance  with  contract 
specification  requirements  rather  than  to  test  the  complete 
find-fix-identification  (ID)-track-target-engage-assess-kill 
chain  for  air-to-air  and  air-to-ground  mission  success. 


The  test  team  originally  designed  WDA  events,  however, 
purposefully  to  gather  weapons  integration  and  fire-control 
performance  using  all  the  mission  systems  required  to 
engage  and  kill  targets  in  the  full  kill  chain.  WDA  events, 
therefore,  became  the  developmental  test  venue  that 
highlighted  the  impact  of  the  backlog  of  deficiencies  created 
by  focusing  prior  testing  only  on  contract  specification 
compliance,  instead  of  readiness  for  combat. 

•  Each  WDA  event  requires  scenario  dry-runs  in  preparation 
for  the  final  end-to-end  event  to  ensure  the  intended 
mission  systems  functionality,  as  well  as  engineering  and 
data  analysis  requirements  (to  support  the  test  centers  and 
weapon  vendors),  are  available  to  complete  the  missile  shot 
or  bomb  drop.  Per  the  approved  TEMP,  the  preparatory 
and  end-to-end  WDA  events  must  be  accomplished  with 
full  mission  systems  functionality,  including  operationally 
realistic  fire  control  and  sensor  performance.  However,  as 
stated  above,  the  program  executed  all  12  of  the  Block  2B 
WDA  events  using  significant  procedural  and  technical 
workarounds  to  compensate  for  the  deficiencies  resident  in 
the  Block  2B  configuration. 

-  Deficiencies  in  the  Block  2B  mission  systems  software 
affecting  the  WDA  events  were  identified  in  fusion, 
radar,  passive  sensors,  identification  friend-or-foe,  EOTS, 
and  the  aircraft  navigation  model.  Deficiencies  in  the 
datalink  systems  also  delayed  completion  of  some  events. 
Developmental  test  team  intervention  was  required  from 
the  control  room  to  overcome  deficiencies  in  order  to 
confirm  surface  target  coordinates,  confirm  actual  air 
targets  among  false  tracks,  and  monitor/advise  regarding 
track  stability  (which  could  not  be  determined  by  the 
pilot).  Overall,  these  deficiencies  continued  to  delay 

the  CY15  WDA  event  schedule  and  compromised  the 
requirement  to  execute  the  missions  with  fully  functional 
integrated  mission  systems.  Obviously,  none  of  this  test 
team  intervention  would  be  possible  in  combat. 

-  The  first  table  on  the  next  page  shows  the  planned  date, 
completion  or  scheduled  date,  and  the  number  of  weeks 
delayed  for  each  of  the  Block  2B  WDA  preparatory  and 
end-to-end  events.  Events  completed  are  shown  with  dates 
in  bold. 

•  The  accumulated  delays  in  the  developmental  testing  WDA 
schedule  have  delayed  the  initiation  of  the  operation  test 
WDA  events.  The  JSF  Operational  Test  Team  (JOTT)  had 
planned  on  starting  their  full  system  integrated  WDA  event 
testing  in  July  2015;  however,  due  to  the  delays  in  delivery 
of  operationally  representative  mission  systems  software, 
coupled  with  delays  in  modifications  of  the  operational  test 
aircraft  to  the  full  Block  2B  configuration,  this  operational 
test  activity  will  not  start  until  CY16.  This  is  six  months 
after  the  program  and  the  Services  fielded  initial  Block  2B 
capability,  and  three  months  later  than  the  JOTT  had  planned 
to  start. 
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BLOCK  2B  WEAPON  ACCURACY  DELIVERY  EVENTS 


WDA 

Number 

Preparatory  Events 

End-to-End  Event 

Weapon 

Planned 

Completed/ 

Scheduled 

Weeks 

Delayed 

Planned 

Completed/ 

Scheduled 

Weeks 

Delayed 

AIM-120 

102 

Sep  13 

Sep  13 

2 

Oct  13 

Oct  13 

2 

112 

Sep  13 

Sep  13 

3 

Oct  13 

Nov  13 

3 

GBU-12 

113 

Sep  13 

Oct  13 

3 

Oct  13 

Oct  13 

0 

GBU-32 

115 

Sep  13 

Nov  13 

6 

Nov  13 

Dec  13 

3 

108 

Oct  13 

Dec  13 

7 

Dec  13 

Feb  14 

12 

110 

Oct  13 

Aug  13 

43 

Dec  13 

Nov  14 

50 

AIM-120 

111 

Dec  13 

Deferred  to 
Block  3F 

- 

Jan  14 

Deferred  to 
Block  3F 

-- 

106 

Dec  13 

Sep  14 

40 

Jan  14 

Nov  14 

43 

May  14 

GBU-31 

114 

Dec  13 

Jun  14 

45 

Feb  14 

Nov  14 

41 

Oct  14 

104 

Feb  14 

Aug  14 

28 

Mar  14 

Deferred  to 

71 

Sep  14 

30 

Block  3i 

107 

Mar  14 

Jun  14 

12 

May  14 

Feb  15 

30 

101 

May  14 

May  14 

17 

Jun  14 

Jan  15 

26 

AIM-120 

Sep  14 

103 

Jun  14 

Mar  14 

-4 

Aug  14 

May  14 

-10 

May  14 

109 

Jul  14 

Jan  14 

-29 

Sep  14 

Mar  14 

-27 

105 

Sep  14 

Deferred  to 
Block  3F 

- 

Oct  14 

Deferred  to 
Block  3F 

- 

1 .  Some  WDA  events  require  more  than  one  preparatory  event. 


BLOCK  31  WEAPON  ACCURACY  DELIVERY  EVENTS 


Weapon 

WDA 

Number 

Preparatory  Events 

End-to-End  Event 

Planned 

Completed/ 

Scheduled 

Weeks 

Delayed 

Planned 

Completed/ 

Scheduled 

Weeks 

Delayed 

AIM-120 

104 

(deferred 
from  2B) 

Feb  14 

Sep  15 

82 

Mar  14 

Sep  15 

78 

201 

May  15 

May  15 

0 

Jun  15 

Jul  15 

3 

204 

Jul  15 

Jul  15 

0 

Aug  15 

Sep  15 

4 

GBU-12 

202 

May  15 

May  15 

0 

May  15 

Aug  15 

11 

GBU-31 

203 

May  15 

May  15 

0 

Jun  15 

Jun  15 

0 

1 .  Some  WDA  events  require  more  than  one  preparatory  event. 


Block  3i 

•  The  program  planned 
that  Block  3i  would  not 
incorporate  any  new 
capability  or  fixes  from  the 
Block  2B  development/fleet 
release.  The  block  3i  WDA 
events  were  capability 
demonstrations  to  confirm 
translation  of  Block  2B 
performance  to  the  Block  3i 
TR-2  hardware.  The  one 
AIM- 120  missile  shot 
scenario  deferred  from 
Block  2B  was  completed  in 
September  2015. 

•  The  table  to  the  right  shows  the  planned  date,  completion 
or  scheduled  date,  and  weeks  delayed  for  each  of  the  WDA 
preparatory  and  end-to-end  events. 

Block  3F 

•  The  Block  3F  weapons  delivery  plan  currently  contains 
48  events  that  will  test  required  Block  3F  capabilities. 
Twenty-nine  of  these  weapon  profiles  accommodate  full 
Block  3F  expanded  envelope  employment  and  systems 

F-35  JSF 


integrated  testing  of  the  GBU-12,  GBU-31/32  JDAM, 

Navy  JSOW,  GBU-39  SDB-1,  AIM-120,  and  AIM-9X. 
Nineteen  of  the  Block  3F  WDA  events  test  air-to-air  and 
air-to-ground  gun  employment  in  all  three  variants  (F-35A 
internal  gun;  F-35B  and  F-35C  external  gun  pod).  Including 
the  two  deferred  events  from  Block  2B  creates  a  total 
of  50  required  weapons  delivery  accuracy  events  to  be 
accomplished  in  approximately  15  months.  These  Block  3F 
events  are  more  complex  than  the  Block  2B  and  3i  events 
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because  of  additional  capability  in  mission  systems  such  as 
advanced  geolocation,  multiple  weapon  events,  enhanced 
radar  modes,  and  expanded  weapons  envelopes  and  loadouts. 
As  will  be  needed  in  combat  employment.  Block  3F  WDA 
events  will  require  reliable  and  stable  target  tracking,  full 
MADL  shoot-list  sharing,  Link  16  capability,  and  predictable 
fusion  performance  in  integrated  systems  operation. 

•  While  the  program  has  instituted  several  process  changes  in 
mission  systems  software  testing,  maintaining  the  necessary 
WDA  event  tempo  to  complete  the  Block  3F  events  will 

be  extremely  challenging.  The  current  build  plans  for  each 
Block  3F  software  version  show  that  the  most  challenging 
scenarios  will  not  be  possible  until  the  final  software 
version.  This  increases  the  likelihood  of  late  discoveries  of 
deficiencies,  as  occurred  during  Block  2B  WDA  testing. 

•  Completing  the  full  set  of  Block  3F  WDA  events  by 
May  2017,  the  planned  end  of  Block  3F  flight  test 
according  to  the  most  recent  program  schedule,  will 
require  an  accomplishment  rate  of  over  3  events  per 
month,  more  than  3  times  the  rate  observed  in  completing 
the  12  Block  2B  WDA  events  (approximately  0.8  events 
per  month).  Extending  by  two  months  to  the  end  of  July 
2017,  as  has  recently  been  briefed  by  the  Program  Office 
as  the  end  of  SDD  flight  test,  is  still  unrealistic.  Unless 
the  accomplishment  rate  increases  over  the  rate  during  the 
Block  2B  testing  period,  completing  all  Block  3F  WDA 
events  will  not  occur  until  November  2021.  In  order  to 
meet  the  schedule  requirements  for  weapon  certification, 
the  Program  Office  has  identified  10  high  priority  WDA 
events  for  the  F-35A  and  5  events  for  the  F-35B  and  F-35C 
that  must  be  accomplished  during  Block  3F  developmental 
testing.  The  program  plans  to  accomplish  the  remaining 
35  events  as  schedule  margin  allows.  The  overall  result  of 
the  WDA  events  must  be  that  the  testing  yields  sufficient 
data  to  evaluate  Block  3F  capabilities.  Deleting  numerous 
WDA  events  places  successful  10T&E  and  combat  capability 
at  significant  risk. 

Static  Structural  and  Durability  Testing 

•  Structural  durability  testing  of  all  variants  using  full-scale 
test  articles  is  ongoing,  with  each  having  completed  at  least 
one  full  lifetime  (8,000  equivalent  flight  hours,  or  EFFl). 

All  variants  are  scheduled  to  complete  three  full  lifetimes 
of  testing  before  the  end  of  SDD;  however,  complete 
teardown,  analyses,  and  Damage  Assessment  and  Damage 
Tolerance  reporting  is  not  scheduled  to  be  completed 

until  August  2019.  The  testing  on  all  variants  has  led  to 
discoveries  requiring  repairs  and  modification  to  production 
designs  and  retrofits  to  fielded  aircraft. 

•  F-35A  durability  test  article  (AJ-1)  completed  the  second 
lifetime  of  testing,  or  16,000  EFH  in  October  2015.  While 
nearing  completion  of  the  second  lifetime,  testing  was  halted 
on  August  13,  2015,  when  strain  gauges  on  the  forward 
lower  flange  of  FS518,  an  internal  wing  structure,  indicated 
deviations  from  previous  trends.  Inspections  showed 
cracking  through  the  thickness  of  the  flange,  so  the  program 


designed  an  interim  repair  to  allow  testing  to  continue  and 
finish  the  second  lifetime. 

•  F-35B  durability  test  article  (BH-1)  completed  11,915  EFH 
by  August  13,  2015,  which  is  3,915  hours  (48.9  percent)  into 
the  second  lifetime.  The  program  completed  the  11,000  hour 
data  review  on  August  5,  2015. 

-  Two  main  wing  carry-through  bulkheads,  FS496  and 
FS472,  are  no  longer  considered  production-representative 
due  to  the  extensive  existing  repairs.  The  program  plans 
to  continue  durability  testing,  repairing  the  bulkheads  as 
necessary,  through  the  second  lifetime  (i.e.,  8,001  through 
16,000  EFH)  which  is  projected  to  be  complete  in 
mid-2016. 

-  Prior  to  CY15,  testing  was  halted  on  September  29,  2013, 
at  9,056  EFH,  when  the  FS496  bulkhead  severed, 
transferred  loads  to,  and  caused  cracking  in  the  adjacent 
three  bulkheads  (FS518,  FS472,  and  FS450).  The 
repairs  and  an  adequacy  review  were  completed  on 
December  17,  2014,  when  the  program  determined  that 
the  test  article  could  continue  testing.  Testing  restarted  on 
January  19,  2015,  after  a  16-month  delay. 

-  The  program  determined  that  several  of  the  cracks 
discovered  from  the  September  2013  pause  at  9,056  EFH 
were  initiated  at  etch  pits.  These  etch  pits  are  created  by 
the  etching  process  required  prior  to  anodizing  the  surface 
of  the  structural  components;  anodizing  is  required  for 
corrosion  protection.  Since  the  cracks  were  not  expected, 
the  program  determined  that  the  etch  pits  were  more 
detrimental  to  fatigue  life  than  the  original  material  design 
suggested.  The  program  is  currently  developing  an 
analysis  path  forward  to  determine  the  effect  on  the  overall 
fatigue  life. 

-  Discoveries  requiring  a  pause  in  testing  during  CY15 
include: 

■  Cracking  in  the  left-  and  right-hand  side  aft  boom 
closeout  frames,  which  are  critical  structural  portions  at 
the  very  aft  of  the  airframe  on  each  side  of  the  engine 
nozzle,  at  9,080  EFH.  The  cracks  were  not  predicted 
by  modeling  and  required  a  three-week  pause  in  testing 
for  repair,  which  consisted  of  a  doubler  (i.e.,  additional 
supporting  element)  as  an  interim  fix  to  allow  testing 

to  continue.  Designs  for  retrofitting  and  cut-in  for 
production  are  under  development. 

■  Damage  to  a  significant  number  of  Electro-Hydraulic 
Actuator  System  (EHAS)  fasteners  and  grommets 

at  9,333  EFH.  The  EHAS  drives  the  aircraft  control 
surfaces  based  on  the  direction  and  demand  input  by  the 
pilot  through  the  control  stick. 

■  Inspections  in  April  2015  revealed  that  cracks  at  four 
previously-identified  web  fastener  holes  near  the 
trunnion  lug  of  the  FS496  bulkhead,  a  component 
integral  to  the  bulkhead  that  supports  the  attachment 
of  the  main  landing  gear  to  the  airframe,  had  grown 
larger.  FS496  was  previously  identified  as  a  life-limited 
part  and  will  be  modified  as  part  of  the  life-limited 
modification  plans  for  production  aircraft  in  Lots  1 
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through  8,  and  a  new  production  design  cut  into  Lot  9 
and  later  lot  aircraft. 

■  Failure  of  the  left  3-Bearing  Swivel  Nozzle  door  uplock 
in  April  2015;  requiring  replacement  prior  to  restarting 
testing  in  May  2015. 

■  Crack  indication  found  at  two  fastener  holes  on  the  left 
side  keel. 

■  Crack  reoccurrence  at  the  Station  3  pylon  at  10,975  EFH. 

■  Cracks  on  the  transition  duct  above  the  vanebox,  a 
component  of  the  lift  fan,  discovered  in  August  2015, 
requiring  the  jacks  that  transmit  loads  to  the  duct  to  be 
disconnected  to  allow  cycling  of  the  rest  of  the  test  article 
to  continue. 

■  During  the  repair  activity  in  September  2015,  a  crack 
was  discovered  in  a  stiffener  on  the  right-hand  side  of  the 
mid-fairing  longeron. 

-  Testing  has  been  paused  since  August  2015  to  allow 

replacement  and  repair  activities;  a  process  estimated  to 

take  five  months.  Testing  is  planned  to  restart  in  January 

2016. 

•  Testing  of  the  F-35C  durability  test  article  (CJ-1)  was  paused 
at  the  end  of  October  2015  when  cracks  were  discovered  in 
both  sides  (i.e.,  the  right-  and  left-hand  sides)  of  one  of  the 
front  wing  spars  after  13,731  EFH  of  testing.  The  Program 
Office  considers  this  to  be  a  significant  finding,  since  the 
wing  spar  is  a  primary  structural  component  and  the  cracking 
was  not  predicted  by  finite  element  modeling.  Root  cause 
analysis  and  options  for  repairing  the  test  article  are  under 
consideration  as  of  the  writing  of  this  report.  Testing  of  the 
second  lifetime  (16,000  EFH)  was  scheduled  to  be  completed 
by  February  1,  2016,  but  discoveries  and  associated  repairs 
over  the  last  year  put  this  testing  behind  schedule. 

-  Additional  discoveries  since  October  2014  include: 

■  Cracking  of  the  BL12  longerons,  left  and  right  sides, 
at  10,806  EFH,  required  a  10-week  pause  in  testing  for 
repairs.  The  effect  to  production  and  retrofit  is  still  to  be 
determined. 

■  Cracks  on  the  FS518  wing  carry-through  lower  bulkhead 
at  11,770  EFH  in  May  2015. 

■  A  crack  at  butt  line  23  on  the  right  hand  side  of  the 
FS496  bulkhead  (initiating  at  a  fastener  hole). 

■  A  crack  was  discovered  during  the  Level-2  inspection 
in  the  FS472  wing  carry- through  bulkhead  after  the 
completion  of  12,000  EFH  in  June  2015.  Repair  work 
was  completed  prior  to  restarting  testing  in  late  August. 

•  The  program  plans  to  use  Laser  Shock  Peening  (LSP),  a 
mechanical  process  designed  to  add  compressive  residual 
stresses  in  the  materials,  in  an  attempt  to  extend  the  lifetime 
of  the  FS496  and  FS472  bulkheads  in  the  F-35B.  The 

first  production  line  cut-in  of  LSP  would  start  with  Lot  11 
F-35B  aircraft.  Earlier  Lot  F-35B  aircraft  will  undergo 
LSP  processing  as  part  of  a  depot  modification.  Testing  is 
proceeding  in  three  phases:  first,  coupon-level  testing  to 
optimize  LSP  parameters;  second,  element-level  testing  to 
validate  LSP  parameters  and  quantify  life  improvement;  and 
third,  testing  of  production  and  retrofit  representative  articles 


to  verify  the  service  life  improvements.  All  three  phases  are 
in  progress,  with  full  qualification  testing  scheduled  to  be 
completed  in  October  2017. 

Verification  Simulation  (VSim) 

•  Due  to  inadequate  leadership  and  management  on  the  part 
of  both  the  Program  Office  and  the  contractor,  the  program 
has  failed  to  develop  and  deliver  an  adequate  Verification 
Simulation  (VSim)  for  use  by  either  the  developmental 
test  team  or  the  JSF  Operational  Test  Team  (JOTT),  as  has 
been  planned  for  the  past  eight  years  and  is  required  in 
the  approved  TEMP.  Neither  the  Program  Office  nor  the 
contractor  has  accorded  VSim  development  the  necessary 
priority,  despite  early  identification  of  requirements 

by  the  JOTT,  $250  Million  in  funding  added  after  the 
Nunn-McCurdy-driven  restructure  of  the  program  in  2010, 
warnings  that  development  and  validation  planning  were  not 
proceeding  in  a  productive  and  timely  manner,  and  recent 
(but  too  late)  intense  senior  management  involvement.  As  a 
result,  VSim  development  is  another  of  several  critical  paths 
to  readiness  for  IOT&E. 

•  The  Program  Office’s  subsequent  decision  in 
September  2015  to  move  the  VSim  to  a  Naval  Air  Systems 
Command  (NAVA1R)  proposal  for  a  government-led  Joint 
Simulation  Environment  (JSE)  will  not  result  in  a  simulation 
with  the  required  capabilities  and  fidelity  in  time  for  F-35 
IOT&E.  Without  a  high-fidelity  simulation,  the  F-35  IOT&E 
will  not  be  able  to  test  the  F-35’s  full  capabilities  against  the 
full  range  of  required  threats  and  scenarios.  Nonetheless, 
because  aircraft  continue  to  be  produced  in  substantial 
quantities  (essentially  all  of  which  require  modifications  and 
retrofits  before  being  used  in  combat),  the  IOT&E  must  be 
conducted  without  further  delay  to  demonstrate  F-35  combat 
effectiveness  under  the  most  realistic  conditions  that  can  be 
obtained.  Therefore,  to  partially  compensate  for  the  lack  of 
a  simulator  test  venue,  the  JOTT  will  now  plan  to  conduct 

a  significant  number  of  additional  open-air  flights  during 
IOT&E,  in  addition  to  those  previously  planned.  In  the 
unlikely  event  a  simulator  is  available  in  time  for  IOT&E, 
the  additional  flights  would  not  be  flown. 

•  VSim  is  a  man-in-the-loop,  mission  systems  software- 
in-the-loop  simulation  developed  to  meet  the  operational 
test  requirements  for  Block  3F  IOT&E.  It  is  also  planned 
by  the  Program  Office  to  be  used  as  a  venue  for  contract 
compliance  verification  prior  to  IOT&E.  It  includes  an 
operating  system  in  which  the  simulation  runs,  a  Battlespace 
Environment  (BSE),  models  of  the  F-35  and  other  supporting 
aircraft,  and  models  of  airborne  and  ground-based  threats. 
After  reviewing  a  plan  for  the  government  to  develop  VSim, 
the  Program  Office  made  the  decision  in  2011  to  have  the 
contractor  develop  the  simulation  instead. 

•  The  Program  Office  began  a  series  of  tests  in  2015  to  ensure 
that  the  simulation  was  stable  and  meeting  the  reduced  set 
of  requirements  for  limited  Block  2B  operational  activities. 
Though  the  contractor’s  BSE  and  operating  system  had 
improved  since  last  year,  deficiencies  in  specific  F-35  sensor 
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models  and  the  lack  of  certain  threat  models  would  have 
limited  the  utility  of  the  VSim  for  Block  2B  operational 
testing,  had  it  occurred.  The  program  elected  instead  to 
provide  a  VSim  capability  for  limited  tactics  development. 
The  Air  Force’s  Air  Combat  Command,  which  is  the  lead  for 
developing  tactics  in  coordination  with  the  other  services, 
planned  two  VSim  events  for  2015. 

-  Air  Combat  Command  completed  the  first  event  in  J  uly 
which  included  one-  and  two-ship  attack  profiles  against 
low  numbers  of  enemy  threats.  This  event  was  planned 
to  inform  the  tactics  manual  that  will  support  IOT&E  and 
the  operational  units,  but  validation  problems  prevented 
detailed  analysis  of  results  (i.e.,  minimum  abort  ranges). 

-  The  second  event,  led  by  the  JOTT  with  Marine  Corps 
pilots  flying,  was  completed  in  October  2015  for  the 
limited  use  of  data  collection  and  mission  rehearsals  to 
support  test  preparation  for  IOT&E.  While  valuable 
lessons  were  learned  by  the  JOTT  and  the  Marine  Corps, 
the  lack  of  accreditation  made  it  impossible  for  the  JOTT 
to  make  assessments  of  F-35  system  performance. 

•  Verification,  Validation,  and  Accreditation  (VV&A)  activity 
completely  stalled  in  2015  and  did  not  come  close  to  making 
the  necessary  progress  towards  even  the  reduced  set  of 
Block  2B  requirements. 

-  Less  than  10  percent  of  the  original  validation  points 
were  collected  from  flight  test  results,  and  a  majority  of 
those  showed  significant  deviations  from  installed  system 
performance.  The  vehicle  systems  model,  which  provides 
the  aircraft  performance  and  flying  qualities  for  the 
simulation,  and  certain  weapons  and  threats  models,  were 
generally  on  track.  However,  mission  systems,  composed 
of  the  sensor  models  and  fusion,  had  limited  validation 
data  and  were  often  unstable  or  not  tuned,  as  required,  to 
represent  the  installed  mission  systems  performance,  as 
measured  in  flight-testing. 

-  The  contractor  and  program  management  failed  to 
intervene  in  time  to  produce  a  simulation  that  met 
even  the  reduced  set  of  user  requirements  for  Block  2B 
and,  although  they  developed  plans  to  increase  VV&A 
productivity,  they  did  not  implement  those  plans  in  time  to 
make  a  tangible  difference  by  the  time  of  this  report.  As 
the  focus  changed  to  Block  3F  and  IOT&E,  the  contractor 
and  the  Program  Office  made  little  progress;  no  VV&A 
plans  materialized,  data  that  had  been  collected  were  still 
stalled  at  the  test  venues  awaiting  review  and  release, 
alternative  data  sources  had  not  yet  been  identified  for  new 
threats,  and  contract  actions  needed  to  complete  VSim  for 
Block  3F  IOT&E  were  not  completed. 

•  In  September  2015,  the  Program  Office  directed  a  change 
in  responsibility  for  VSim  implementation,  reassigning 
the  responsibility  from  the  contractor,  Lockheed  Martin, 
to  a  government  team  led  primarily  by  NAVAIR.  This 
was  triggered  by  a  large  increase  in  the  contractor’s  prior 
proposed  cost  to  complete  VSim,  a  cost  increase  which 
included  work  that  should  already  have  been  completed  in 
Block  2B  and  mitigations  intended  to  overcome  prior  low 


productivity.  The  path  to  provide  an  adequate  validation  of 
the  simulation  for  Block  3F  IOT&E  carries  risk,  regardless 
of  who  is  responsible  for  the  implementation  of  the 
simulation.  That  risk  was  increased  by  the  Program  Office’s 
decision  to  move  the  simulation  into  a  government  controlled 
(non-proprietary)  facility  and  simulation  environment. 

After  analyzing  the  steps  needed  to  actually  implement  the 
Program  Office’s  decision  to  move  the  VSim  to  the  JSE, 
it  is  clear  that  the  JSE  will  not  be  ready,  with  the  required 
capabilities  and  fidelity,  in  time  for  F-35  IOT&E  in  2018. 

It  is  also  clear  that  both  NAVAIR  and  the  Program  Office 
significantly  underestimated  the  scope  of  work,  the  cost,  and 
the  time  required  to  replace  Lockheed  Martin’s  proprietary 
BSE  with  the  JSE  while  integrating  and  validating  the 
required  high-fidelity  models  for  the  F-35,  threats,  friendly 
forces,  and  other  elements  of  the  combat  environment. 

-  The  JSE  proposal  abandons  the  BSE  that  is  currently 
running  F-35  Block  2B. 

-  The  JSE  proposal  does  not  address  longstanding 
unresolved  issues  with  VSim,  including  the  ability  of  the 
program  to  produce  validation  data  from  flight  test,  to 
analyze  and  report  comparisons  of  that  data  with  VSim 
performance,  and  to  “tune”  VSim  to  match  the  installed 
system  performance  demonstrated  in  flight-testing. 

-  While  the  JSE  might  eventually  reach  the  required  level 
of  fidelity,  it  will  not  be  ready  in  time  for  IOT&E  since 
the  government  team  must  re-integrate  into  the  JSE  the 
highly  detailed  models  of  the  F-35  aircraft  and  sensors, 
and  additional  threat  models  that  the  contractor  has 
“hand-built”  over  several  years. 

-  The  current  VSim  F-35  aircraft  and  sensor  models  interact 
directly  with  both  the  BSE  and  the  current  contractor’s 
operating  system.  A  transition  to  the  JSE  will  require 

a  re-architecture  of  these  models  before  they  can  be 
integrated  into  a  different  environment.  The  need  to  do 
this,  along  with  the  costs  of  contractor  support  for  the 
necessary  software  models  and  interfaces,  will  overcome 
the  claims  of  cost  savings  in  NAVAIR’s  proposal. 

-  The  highly  integrated  and  realistic  manned  “red  air” 
simulations  in  VSim,  which  were  inherited  from  other 
government  simulations,  cannot  be  replicated  in  the 
limited  time  remaining  before  IOT&E. 

-  The  large  savings  estimates  claimed  by  NAVAIR 
as  the  basis  for  their  JSE  proposal  are  not  credible, 
and,  the  government  team’s  most  recent  estimates  for 
completion  of  the  JSE  have  grown  substantially  from 
its  initial  estimate.  Nearly  all  the  costs  associated  with 
completing  VSim  in  its  current  form  would  also  transfer 
directly  to  JSE,  with  significant  additional  delays  and 
risk.  Any  potential  savings  in  the  remaining  costs  from 
government-led  integration  are  far  outweighed  by  the 
additional  costs  associated  with  upgrading  or  building  new 
facilities,  upgrading  or  replacing  the  BSE,  re-hosting  the 
F-35  on  government  infrastructure,  and  paying  Lockheed 
Martin  to  build  interfaces  between  their  F-35  models  and 
the  JSE. 
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-  The  JSE  proposal  adds  significant  work  and  schedule  risk 
to  the  contractor’s  ability  to  deliver  a  functioning  and 
validated  Block  3F  aircraft  model  in  time  for  IOT&E. 
Besides  being  required  to  complete  integration  of 
Block  3F  capabilities,  validate  the  simulation,  and  tune 
the  sensor  models  to  installed  system  performance,  the 
contractor  must  also  simultaneously  assist  the  government 
in  designing  new  interfaces  and  re-hosting  the  F-35  and 
hand-built  threat  models  into  the  JSE  to  all  run  together  in 
real-time  so  they  can  be  validated  and  accredited. 

-  Abandoning  VSim  also  affects  the  F-22  program,  as  the 
various  weapons  and  threat  models  being  developed  were 
planned  to  be  reused  between  the  two  programs.  The 
upcoming  F-22  Block  3.2B  IOT&E  depends  on  the  BSE 
currently  in  development. 

•  For  the  reasons  listed  above,  the  Program  Office’s  decision  to 
pursue  the  NAVAIR-proposed  JSE,  without  the  concurrence 
of  the  operational  test  agencies  (OTAs)  or  DOT&E,  will 
clearly  not  provide  an  accredited  simulation  in  time  for  F-35 
IOT&E,  and  the  OTAs  have  clearly  expressed  their  concerns 
regarding  the  risks  posed  to  the  IOT&E  by  the  lack  of  VSim. 
Nonetheless,  so  as  not  to  delay  IOT&E  any  further  while 
substantial  numbers  of  aircraft  are  being  produced,  DOT&E 
and  the  OTAs  have  agreed  on  the  need  to  now  plan  for  the 
F-35  IOT&E  assuming  a  simulator  will  not  be  available. 

This  will  require  flying  substantial  additional  open-air  flights 
for  tactics  development,  mission  rehearsal,  and  evaluation 

of  combat  effectiveness  relative  to  previous  plans  for  using 
VSim.  Even  with  these  additional  flights,  some  testing 
previously  planned  against  large-scale,  real-world  threat 
scenarios  in  VSim  will  no  longer  be  possible. 

Live  Fire  Test  and  Evaluation  (LFT&E) 

F-35C  Full-Scale  Fuel  Ingestion  Tolerance  Vulnerability 

Assessment 

•  The  F-35  LFT&E  Program  completed  the  F-35C  full-scale, 
fuel  ingestion  tolerance  test  series.  The  Navy’s  Weapons 
Survivability  Laboratory  (WSL)  in  China  Lake,  California, 
executed  four  tests  events  using  the  CG:0001  test  article. 

Two  of  the  test  events  were  conducted  with  a  Pratt  and 
Whitney  F-135  initial  flight  release  (IFR)-configured  engine 
installed  in  the  aircraft.  A  preliminary  review  of  the  results 
indicates  that: 

-  The  FI 35  IFR-configured  engine  is  tolerant  of  fuel 
ingestion  caused  by  single  missile-warhead  fragment 
impacts  in  the  F 1  fuel  tank.  The  threat-induced  fuel 
discharge  into  the  engine  inlet  caused  temporary  increases 
in  the  nominal  engine  temperature,  but  did  not  result  in 
any  engine  stalls  or  long-term  damage. 

-  Missile  fragment-induced  damage  is  consistent  with 
predictions  and  the  tanks  are  tolerant  of  single-fragment 
impacts.  The  threat-induced  damage  to  the  FI  fuel 
tank  caused  fuel  leak  rates  that  are  consistent  with  tests 
conducted  in  FY07  using  flat  panels. 


PAO  Shut-Off  Valve 

•  The  program  has  not  provided  an  official  decision  to 
reinstate  this  vulnerability  reduction  feature.  There  has 
been  no  activity  on  the  development  of  the  PAO-shut-off 
valve  technical  solution  to  meet  criteria  developed  from 
2011  live  fire  test  results.  As  stated  in  several  previous 
reports,  this  aggregate,  2-pound  vulnerability  reduction 
feature,  if  installed,  would  reduce  the  probability  of  pilot 
incapacitation,  decrease  overall  F-35  vulnerability,  and 
prevent  the  program  from  failing  one  of  its  vulnerability 
requirements. 

Fuel  Tank  Ullage  Inerting  System  and  Lightning  Protection 

•  The  program  verified  the  ullage  inerting  design  changes, 
including  a  new  pressurization  and  ventilation  control 
valve,  wash  lines  to  the  siphon  tanks,  and  an  external  wash 
line,  and  demonstrated  improved  inerting  performance 

in  F-35B  fuel  system  simulator  tests.  Apreliminary  data 
review  demonstrated  that  the  system  pressurized  the  fuel 
tank  with  nitrogen  enriched  air  (NEA)  while  maintaining 
pressure  differentials  within  design  specifications  during 
all  mission  profiles  in  the  simulator,  including  rapid  dives. 
The  Program  Office  will  complete  and  document  a  detailed 
data  review  and  analyses  that  evaluate  NEA  distribution  and 
inerting  uniformity  between  different  fuel  tanks  and  within 
partitioned  fuel  tanks. 

•  The  program  developed  a  computational  model  to  predict 
inerting  performance  in  the  aircraft  based  on  the  F-35B 
simulator  test  results.  Patuxent  River  Naval  Air  Station 
completed  the  ground  inerting  test  on  a  developmental  test 
F-35B  aircraft  to  verify  the  inerting  model.  Preliminary 
analyses  of  the  results  indicate  that  there  is  good  correlation 
between  the  ground  inerting  test  and  the  F-35B  fuel  system 
simulator.  The  program  will  use  this  model,  in  conjunction 
with  the  completed  F-35A  and  F-35C  ground  tests,  to  assess 
the  ullage  inerting  effectiveness  for  all  three  variants.  The 
confidence  in  the  final  design’s  effectiveness  will  have  to 
be  reassessed  after  the  deficiencies  uncovered  in  the  aircraft 
ground  and  flight  tests,  including  small  uninerted  fuel  tank 
ullage  spaces,  have  been  fully  resolved. 

•  When  effective,  ullage  inerting  protects  the  fuel  tanks  from 
not  just  threat-induced  damage  but  also  lightning-induced 
damage.  The  ullage  inerting  system  does  not  protect  any 
other  components  or  systems  from  lightning-induced 
damage. 

•  The  program  has  made  progress  completing  lightning 
tolerance  qualification  testing  for  line-replaceable  units 
needed  to  protect  the  remaining  aircraft  systems  from 
lightning-induced  currents.  Lightning  tolerance  tests  using 
electrical  current  injection  tests  are  ongoing,  and  the  program 
expects  to  complete  the  tests  by  2QFY16. 

Vulnerability  to  Unconventional  Threats 

•  The  full-up,  system-level  chemical-biological 
decontamination  test  on  an  SDD  aircraft  planned  for 
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4QFY16  at  Edwards  AFB  is  supported  by  two  risk-reduction 
events: 

-  A  System  Integration  Demonstration  of  the  proposed 
decontamination  equipment  and  shelter  was  conducted  on 
an  F-16  test  article  during  1QFY15  at  Edwards  AFB  to 
simulate  both  hot  air  chemical  and  hot/humid  air  biological 
decontamination  operations.  Extensive  undesirable 
condensation  inside  the  shelter  and  on  the  test  article 
during  the  hot/humid  air  biological  decontamination  event 
indicated  the  need  for  process  and  shelter  modifications. 

-  A  demonstration  of  an  improved  shelter  is  planned  for 
2QFY16  to  demonstrate  that  a  modified  system  process 
and  better  insulated  shelter  can  maintain  adequate 
temperature  and  humidity  control  inside  the  shelter,  even 
in  a  cold-weather  environment. 

•  The  test  plan  to  assess  chemical  and  biological 
decontamination  of  pilot  protective  equipment  is  not 
adequate.  Compatibility  testing  of  protective  ensembles 
and  masks  has  shown  that  the  materials  survive  exposure 
to  chemical  agents  and  decontamination  materials  and 
processes,  but  the  program  has  neither  tested  nor  provided 
plans  for  testing  the  Helmet  Mounted  Display  Systems 
(HMDS)  currently  being  fielded.  Generation  II  HMDS 
compatibilities  were  determined  by  analysis,  comparing 
HMDS  materials  with  those  in  an  extensive  DOD  aerospace 
materials  database.  A  similar  analysis  is  planned  for  the 
Generation  III  HMDS  design.  However,  even  if  material 
compatibilities  were  understood,  there  are  no  plans  to 
demonstrate  a  process  that  could  adequately  decontaminate 
either  HMDS  from  chemical  and  biological  agents. 

•  The  Joint  Program  Executive  Office  for  Chemical 
and  Biological  Defense  approved  initial  production 
of  the  F-35  variant  of  the  Joint  Service  Aircrew  Mask 
(JSAM-JSF)  during  1QFY16.  This  office  and  the  F-35 
Joint  Program  Office  are  integrating  the  JSAM-JSF  with  the 
Helmet-Mounted  Display,  which  is  undergoing  Safety  of 
Flight  testing. 

•  The  Navy  evaluated  an  F-35B  aircraft  to  the  EMP  threat 
level  defined  in  MIL-STD-2169B.  Follow-on  tests  on  other 
variants  of  the  aircraft,  including  a  test  series  to  evaluate  any 
Block  3F  hardware/software  changes,  are  planned  for  FY16. 

Gun  Ammunition  Lethality  and  Vulnerability 

•  The  program  completed  the  terminal  ballistic  testing  of  the 
PGU-47  APEX  round  against  a  range  of  target-representative 
material  plates  and  plate  arrays.  Preliminary  test 
observations  indicated  expected  high  levels  of  fragmentation 
when  passing  through  multiple  layer,  thin  steel  or  aluminum 
targets,  along  with  a  deep  penetration  through  more  than  an 
inch  of  rolled  homogeneous  armor  steel  by  the  nose  of  the 
penetrator.  The  program  will  evaluate  the  effect  of  these  data 
on  the  ammunition  lethality  assessment. 

•  The  780th  Test  Squadron  at  Eglin  AFB  has  completed  the 
ground-based  Frangible  Armor  Piercing  (FAP)  and  initiated 
the  PGU-32  lethality  tests.  The  APEX  rounds  will  be  tested 
in  FY16  against  a  similar  range  of  targets,  including  armored 
and  technical  vehicles,  aircraft,  and  personnel  in  the  open. 


Ground-based  lethality  tests  for  the  FAP  showed  expected 
high  levels  of  penetration  against  all  targets,  with  slightly 
less  internal  target  fragmentation  than  originally  anticipated, 
and  low  levels  of  lethality  against  personnel  in  the  open 
(unless  impacted  directly).  The  program  will  determine  the 
effect  of  these  data  on  the  ammunition  lethality  assessment. 

•  Per  the  current  mission  systems  software  schedule,  the 
weapons  integration  characterization  of  the  gun  and  sight 
systems  will  not  be  ready  for  the  air-to-ground  gun  strafe 
lethality  tests  until  1QFY17.  Strafing  targets  will  include 
a  small  boat,  light  armored  vehicle  and  technical  vehicle 
(pickup  truck),  one  each  for  each  round  type  tested. 

Because  the  APEX  round  is  not  currently  a  part  of  the 
program  of  record,  funding  for  developmental  or  operational 
air-to-ground  flight  testing  of  the  APEX  round  is  not  planned 
at  this  time. 

Operational  Suitability 

•  Operational  suitability  of  all  variants  continues  to  be  less 
than  desired  by  the  Services,  and  relies  heavily  on  contractor 
support  and  workarounds  that  woidd  be  difficult  to  employ  in 
a  combat  environment.  Almost  all  measures  of  performance 
have  improved  over  the  past  year,  but  most  continue  to  be 
below  their  interim  goals  to  achieve  acceptable  suitability 

by  the  time  the  fleet  accrues  200,000  flight  hours,  the 
benchmark  set  by  the  program  and  defined  in  the  Operational 
Requirements  Document  (ORD)  for  the  aircraft  to  meet 
reliability  and  maintainability  requirements.  This  level  of 
maturity  is  further  stipulated  as  75,000  flight  hours  for  the 
F-35A,  75,000  flight  hours  for  the  F-35B,  and  50,000  flight 
hours  for  the  F-35C. 

-  Aircraft  fleet-wide  availability  averaged  5 1  percent  for 
12  months  ending  October  2015,  compared  to  a  goal  of 
60  percent. 

-  Availability  had  been  in  mid-30s  to  low-40s  percent 
for  the  2-year  period  ending  September  2014.  Monthly 
availability  jumped  12  percent  to  51  percent  by  the  end 
of  October  2014,  one  of  the  largest  month-to-month 
spikes  in  program  history,  and  then  peaked  at  56  percent 
in  December  2014.  Since  then  it  has  remained  relatively 
flat,  centering  around  50  percent,  although  it  achieved 
56  percent  again  in  September  2015.  The  significant 
improvement  that  occurred  around  October  2014  was  due 
in  roughly  equal  measure  to  a  reduction  in  the  time  aircraft 
were  undergoing  maintenance  and  a  reduction  in  the  time 
aircraft  were  awaiting  spare  parts  from  the  supply  system. 
The  aircraft  systems  that  showed  the  greatest  decreases 
(improvement)  in  maintenance  downtime  during  the  month 
of  October  2014  were  the  engine  and  the  ejection  seat. 

-  It  would  be  incorrect  to  attribute  the  still-low  availability 
the  F-35  fleet  has  exhibited  in  2015,  specifically  the  failure 
to  meet  the  goal  of  60  percent  availability,  solely  to  issues 
stemming  from  the  additional  engine  inspections  required 
since  the  June  2014  engine  failure  on  AF-27.  Availability 
did  drop  immediately  after  the  engine  failure,  partly  due 
to  these  inspections,  but  has  since  recovered  to  pre-engine 
failure  levels,  and  improved  only  slightly  from  there  when 
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considered  as  a  long-term  trend.  For  the  three  months 
ending  October  2015,  the  fleet  was  down  for  the  3rd  Stage 
Integrally  Bladed  Rotor  (1BR)  inspections — required  due 
to  the  engine  failure — less  than  1  percent  of  the  time. 

-  Measures  of  reliability  that  have  ORD  requirement 
thresholds  have  improved  since  last  year,  but  eight  of 
nine  measures  are  still  below  program  target  values  for 
the  current  stage  of  development,  although  two  are  within 
5  percent  of  their  interim  goal;  one — F-35B  Mean  Flight 
Hours  Between  Maintenance  Events  (Unscheduled) — is 
above  its  target  value.  In  addition  to  the  nine  ORD 
metrics,  there  are  three  contract  specification  metrics, 

Mean  Flight  Hour  Between  Failures  scored  as  “design 
controllable”  (one  for  each  variant).  Design  controllable 
failures  are  equipment  failures  due  to  design  flaws 
considered  to  be  the  fault  of  the  contractor,  such  as 
components  not  withstanding  stresses  expected  to  be  found 
in  the  normal  operational  environment.  It  does  not  include 
failures  caused  by  improper  maintenance,  or  caused  by 
circumstances  unique  to  flight  test.  This  metric  continues 
to  see  the  highest  rate  of  growth,  and  for  this  metric  all 
three  variants  are  currently  above  program  target  values 
for  this  stage  in  development. 

-  Although  reliability,  as  measured  by  the  reduced 
occurrence  of  design  controllable  failures,  has  shown 
strong  growth,  this  has  only  translated  into  relatively 
minor  increases  in  availability  for  several  reasons.  These 
reasons  include  the  influences  of  a  large  amount  of  time 
spent  on  scheduled  maintenance,  downtime  to  incorporate 
required  modifications,  waiting  longer  for  spare  parts  than 
planned,  and  potentially  longer-than-expected  repair  times, 
especially  if  units  have  to  submit  Action  Requests  (ARs) 
for  instructions  on  repairs  with  no  written  procedures  yet 
available.  Finally,  aircraft  in  the  field  become  unavailable 
for  failures  not  scored  as  design  controllable  as  well.  All 
of  these  factors  affect  the  final  availability  rate  the  fleet 
achieves  at  any  given  time,  in  addition  to  the  effect  of 
improved  reliability. 

-  F-35  aircraft  spent  21  percent  more  time  than  intended 
down  for  maintenance,  and  waited  for  parts  from  supply 
for  5 1  percent  longer  than  the  program  targeted.  At  any 
given  time,  from  l-in-10  to  l-in-5  aircraft  were  in  a  depot 
facility  or  depot  status  for  major  re-work  or  planned 
upgrades,  and  of  the  fleet  that  remained  in  the  field,  on 
average,  only  half  were  able  to  fly  all  missions  of  even  a 
limited  capability  set. 

Accurate  suitability  measures  rely  on  adjudicated  data 
from  fielded  operating  units.  A  Joint  Reliability  and 
Maintainability  Evaluation  Team  (JRMET),  composed 
of  representatives  from  the  Program  Office,  the  JOTT, 
the  contractor  (Lockheed  Martin),  and  Pratt  and  Whitney 
(for  engine  records),  reviews  maintenance  data  to 
ensure  consistency  and  accuracy  for  reporting  measures; 
government  representatives  chair  the  team.  However,  the 
Lockheed  Martin  database  that  stores  the  maintenance 
data,  known  as  the  Failure  Reporting  and  Corrective  Action 


System  (FRACAS),  is  not  in  compliance  with  U.S.  Cyber 
Command  information  assurance  policies  implemented  in 
August  2015.  Because  of  this  non-compliance,  government 
personnel  have  not  been  able  to  access  the  database  via 
government  networks,  preventing  the  JRMET  from  holding 
the  planned  reviews  of  maintenance  records.  As  a  result,  the 
Program  Office  has  not  been  able  to  produce  Reliability  and 
Maintainability  (R&M)  metrics  from  JRMET-adjudicated 
data  since  the  implementation  of  the  policy.  The  most 
current  R&M  metrics  available  for  this  report  are  from  the 
three-month  rolling  window  ending  in  May  2015.  The 
Program  Office  is  investigating  workarounds  to  enable  the 
JRMET  to  resume  regular  reviews  of  maintenance  records 
until  Lockheed  Martin  can  bring  the  FRACAS  database  into 
compliance. 

F-35  Fleet  Availability 

•  Aircraft  availability  is  determined  by  measuring  the 
percent  of  time  individual  aircraft  are  in  an  “available” 
status,  aggregated  over  a  reporting  period  (e.g.,  monthly). 

The  program  assigns  aircraft  that  are  not  available  to  one 
of  three  categories  of  status:  Not  Mission  Capable  for 
Maintenance  (NMC-M);  Not  Mission  Capable  for  Supply 
(NMC-S);  and  Depot  status. 

-  Program  goals  for  these  “not  available”  categories 
have  remained  unchanged  since  2014,  at  15  percent 
for  NMC-M,  10  percent  for  NMC-S,  and  15  percent  of 
the  fleet  in  depot  status.  Depot  status  is  primarily  for 
executing  the  modification  program  to  bring  currently 
fielded  aircraft  closer  to  their  expected  airframe  structural 
lifespans  of  8,000  flight  hours  and  to  incorporate  additional 
mission  capability.  The  majority  of  aircraft  in  depot  status 
are  located  at  dedicated  depot  facilities  for  scheduled 
modification  periods  that  can  last  several  months,  and  they 
are  not  part  of  the  operational  or  training  fleet  during  this 
time.  A  small  portion  of  depot  status  can  occur  in  the  field 
when  depot  field  teams  conduct  a  modification  at  a  main 
operating  base,  or  affect  repairs  beyond  the  capability  of 
the  local  maintenance  unit. 

-  These  three  “not  available”  category  goals  sum  to 

40  percent,  leaving  a  targeted  fleet-wide  goal  of  60  percent 
availability  for  2015.  At  the  time  of  this  report,  this 
availability  goal  extended  uniformly  to  the  individual 
variants,  with  each  variant  having  a  target  of  60  percent 
availability  as  well.  For  a  period  during  2015,  however, 
the  program  set  variant-specific  availability  goals  to 
account  for  the  fact  that  the  variants  were  cycling  through 
the  depots  at  different  rates.  A  particularly  large  portion 
of  the  F-35B  fleet  was  in  depot  in  early  2015  to  prepare 
aircraft  for  Marine  Corps  IOC  declaration,  for  example. 
From  February  to  August  2015,  the  variant-specific 
availability  goals  were  reported  as  65  percent  for  the 
F-35A,  45  percent  for  the  F-35B,  and  70  percent  for  the 
F-35C,  while  the  total  fleet  availability  goal  remained 
60  percent. 
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•  Aircraft  monthly  availability  averaged  5 1  percent  for  the 
12-month  period  ending  October  2015  in  the  training  and 
operational  fleets.  This  is  an  increase  over  the  37  percent 
availability  reported  in  both  of  the  previous  two  DOT&E 
Annual  Reports  from  FY13  and  FY14. 

•  However,  in  no  month  did  the  fleet  exceed  its  goal  of 
60  percent  availability.  In  several  months,  individual 
variants  beat  either  the  60  percent  goal,  or  their  at-the-time 
variant-specific  goal.  The  F-35A  achieved  63  percent 
availability  in  December  2014,  but  never  surpassed 

65  percent.  The  F-35C  was  above  60  percent  availability 
from  November  2014  to  June  2015,  and  again  in 
September  and  October  2015,  and  was  above  70  percent 
in  four  of  these  months.  The  F-35B  was  above  45  percent 
availability  in  only  one  month,  October  2015,  when  it 
achieved  48  percent.  This  was  after  the  program  returned  its 
variant-specific  availability  target  to  60  percent. 

•  The  table  below  summarizes  aircraft  availability  rates 
by  operating  location  for  the  12-month  period  ending 
October  2015.  The  first  column  indicates  the  average 
availability  achieved  for  the  whole  period,  while  the 
maximum  and  minimum  columns  represent  the  range  of 
monthly  availabilities  reported  over  the  period.  The  number 
of  aircraft  assigned  at  the  end  of  the  reporting  period  is 
shown  as  an  indicator  of  potential  variance  in  the  rates.  Sites 
are  arranged  in  order  of  when  each  site  began  operation  of 
any  variant  of  the  F-35,  and  then  arranged  by  variant  for 
sites  operating  more  than  one  variant.  In  February  2015,  the 
Marine  Corps  terminated  operations  of  the  F-35B  at  Eglin 
AFB  and  transferred  the  bulk  of  the  aircraft  from  that  site  to 
Marine  Corps  Air  Station  (MCAS)  Beaufort,  South  Carolina. 
As  a  result,  the  number  of  F-35B  aircraft  assigned  to  Eglin 
AFB  as  of  September  2015  was  zero. 
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Operational 

Site 

Average 

Maximum 

Minimum 

Aircraft 

Assigned2 

Whole  Fleet 

51% 

56% 

46% 

134 

Eglin  F-35  A 

55% 

62% 

39% 

25 

Eglin  F-35B3 

43% 

48% 

26% 

0 

Eglin  F-35C 

66% 

79% 

57% 

17 

Yuma  F-35B 

39% 

62% 

16% 

17 

Edwards  F-35A 

32% 

66% 

17% 

8 

Edwards  F-35B4 

19% 

27% 

0% 

6 

Nellis  F-35A 

51% 

77% 

33% 

10 

Luke  F-35A 

62% 

75% 

50% 

30 

Beaufort  F-35B5 

46% 

60% 

24% 

18 

Hill  F-35A6 

80% 

81% 

79% 

3 

1.  Data  do  not  include  SDD  aircraft. 

2.  Aircraft  assigned  at  the  end  of  October  201 5. 
3.  Eglin  AFB  F-35B  ended  operations  in  February  201 5. 
4.  Edwards  AFB  F-35B  operational  test  operations  began  in  October  2014. 

5.  Beaufort  MCAS  F-35B  operations  began  in  July  201 4. 

6.  Hill  AFB  F-35A  operations  began  September  201 5. 

-  Statistical  trend  analysis  of  the  monthly  fleet  availability 
rates  from  August  2012  through  October  2015  showed 


a  weak  rate  of  improvement  of  approximately  5  percent 
growth  per  year  over  this  period,  but  the  growth  was 
not  consistent.  For  example,  from  August  2012  through 
September  2014,  availability  was  relatively  flat  and  never 
greater  than  46  percent,  but  from  September  2014  through 
December  2014,  it  rose  relatively  quickly  month-on-month 
to  peak  at  56  percent  in  December.  Availability  then 
dropped  a  bit,  and  remained  near  50  percent  through 
October  2015  with  no  increasing  trend  toward  the  goal  of 
60  percent. 

Due  to  concurrency,  the  practice  of  producing  operational 
aircraft  before  the  program  has  completed  development 
and  finalized  the  aircraft  design,  the  Services  must  send  the 
current  fleet  of  F-35  aircraft  to  depot  facilities  to  receive 
modifications  that  have  been  designed  since  they  were 
originally  manufactured.  Some  of  these  modifications 
are  driven  by  faults  in  the  original  design  that  were  not 
discovered  until  after  production  had  started,  such  as  major 
structural  components  that  break  due  to  fatigue  before  their 
intended  lifespan,  and  others  are  driven  by  the  continuing 
improvement  of  the  design  of  combat  capabilities  that 
were  known  to  be  lacking  when  the  aircraft  were  first  built. 
This  “concurrency  tax”  causes  the  program  to  expend 
resources  to  send  aircraft  for  major  re-work,  often  multiple 
times,  to  keep  up  with  the  aircraft  design  as  it  progresses. 
Since  System  Development  and  Demonstration  (SDD) 
will  continue  to  2017,  and  by  then  the  program  will  have 
delivered  nearly  200  aircraft  to  the  U.S.  Services  in  other 
than  the  3F  configuration,  the  depot  modification  program 
and  its  associated  concurrency  burden  will  be  with  the 
Services  for  years  to  come. 

Sending  aircraft  to  depot  facilities  for  several  months 
at  a  time  to  bring  them  up  to  Block  2B  capability  and 
life  limits,  and  eventually  to  3F  configuration,  reduces 
the  number  of  aircraft  at  field  sites  and  thus  decreases 
fleet  availability.  For  the  12-month  period  ending  in 
October  2015,  the  proportion  of  fleet  in  depot  status 
averaged  16  percent.  The  depot  percentage  generally 
increased  slowly  at  first,  reaching  a  maximum  value  of  19 
percent  for  the  month  of  May  2015,  and  then  started  to 
decline  around  summer  2015.  The  depot  inductions  were 
largely  in  support  of  modifying  aircraft  to  the  Block  2B 
configuration  for  the  Marine  Corps  IOC  declaration  at  the 
end  of  July  2015. 

Current  program  plans  indicate  the  proportion  of  the  fleet 
in  depot  will  remain  between  10  and  15  percent  throughout 
CY16.  Projections  of  depot  rates  beyond  2016  are 
difficult,  since  testing  and  development  are  ongoing.  The 
program  does  not  yet  know  the  full  suite  of  modifications 
that  will  be  necessary  to  bring  currently  produced  aircraft 
up  to  the  envisioned  final  Block  3F  configuration. 

To  examine  the  suitability  performance  of  fielded  aircraft, 
regardless  of  how  many  are  in  the  depot,  the  program 
reports  on  the  Mission  Capable  and  Full  Mission  Capable 
(FMC)  rates  for  the  F-35  fleet.  The  Mission  Capable  rate 
represents  the  proportion  of  the  fleet  that  is  not  in  depot 
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status  and  that  is  ready  to  fly  any  type  of  mission  (as 
opposed  to  all  mission  types).  This  rate  includes  aircraft 
that  are  only  capable  of  flying  training  flights,  however, 
and  not  necessarily  a  combat  mission.  Aircraft  averaged 
65  percent  for  the  12-month  window  considering  all 
variants. 

-  The  FMC  rate  calculates  only  the  proportion  of  aircraft 
not  in  depot  status  that  are  capable  of  flying  all  assigned 
missions  and  can  give  a  better  view  into  the  potential 
combat  capability  available  to  the  field.  It  averaged 

46  percent  for  the  12-month  window  considering  all 
variants,  but  started  to  drop  steadily  from  a  peak  of 
62  percent  achieved  in  December  2014,  reaching  a 
minimum  value  of  32  percent  in  October  2015.  The 
rate  declined  for  8  of  the  10  months  from  January  to 
October  2015. 

-  The  monthly  NMC-M  rate  averaged  1 8  percent  over 
the  period,  and  exhibited  the  most  variability  of  the 
non-available  status  categories.  The  NMC-M  rate  started 
out  at  17  percent  in  November  2014,  was  as  high  as 

24  percent  in  August  2015,  and  as  low  as  14  percent  in 
September  2015.  The  Program  Office  set  a  threshold 
goal  of  15  percent  for  2015,  but  the  fluctuations  in 
month-to-month  rates  make  it  difficult  to  determine 
whether  the  goal  for  NMC-M  can  be  achieved  for  a 
sustained  period. 

-  Modifying  aircraft  also  affects  the  NMC-M  rate.  Squadron 
maintainers,  instead  of  the  depot,  are  tasked  to  complete 

a  portion  of  the  required  modifications  by  accomplishing 
Time  Compliance  Technical  Directives  (TCTDs).  The 
“time  compliance”  requirements  for  these  directives  vary, 
normally  allowing  the  aircraft  to  be  operated  without 
the  modification  in  the  interim  and  permit  maintenance 
personnel  to  work  the  directive  as  able.  While  maintainers 
accomplish  these  TCTDs,  the  aircraft  are  logged  as 
NMC-M  status.  Incorporating  these  TCTDs  will  drive 
the  NMC-M  rate  up  (worse)  until  these  remaining 
modifications  are  completed.  Publishing  and  fielding  new 
TCTDs  is  expected  for  a  program  under  development 
and  is  needed  to  see  improvement  in  reliability  and 
maintainability. 

-  The  NMC-S  rate  averaged  15  percent  and  showed  little 
trend,  either  up  or  down,  over  the  period.  The  NMC-S 
rate  started  at  15  percent  in  November  2014  and  ended 
at  16  percent  in  October  2015,  ranging  from  between 
12  to  19  percent  in  the  months  between.  The  Program 
Office  set  a  threshold  goal  of  10  percent  for  2015,  but  the 
NMC-S  trend  is  not  currently  on  track  to  achieve  this. 

-  Modifying  aircraft  also  has  an  effect  on  the  NMC-S 
rate.  Parts  are  taken  from  aircraft  in  depot  status  at  the 
dedicated  modification  facilities  in  order  to  provide 
replacements  for  failed  parts  in  the  field,  a  process 
known  as  depot  cannibalization.  This  usually  occurs 
when  replacement  parts  are  not  otherwise  available 
from  normal  supply  channels  or  stocks  of  spare  parts 
on  base.  With  the  large  number  of  aircraft  in  depot 


status,  the  program  may  have  been  able  to  improve  the 
NMC-S  rate  by  using  depot  cannibalizations,  instead  of 
procuring  more  spare  parts,  or  reducing  the  failure  rate 
of  parts  installed  in  aircraft,  or  improving  how  quickly 
failed  parts  are  repaired  and  returned  to  circulation.  If 
the  Services  endeavor  to  bring  all  of  the  early  lot  aircraft 
into  the  Block  3F  configuration,  the  program  will  continue 
to  have  an  extensive  modification  program  for  several 
years.  While  this  will  continue  to  provide  opportunities 
for  depot  cannibalizations  during  that  time,  once  the  3F 
modifications  are  complete,  there  will  be  fewer  aircraft  in 
the  depot  serving  as  spare  parts  sources  and  more  in  the 
field  requiring  parts  support.  If  demand  for  spare  parts 
remains  high,  this  will  put  pressure  on  the  supply  system 
to  keep  up  with  demand  without  depot  cannibalization  as  a 
source. 

-  Low  availability  rates  are  preventing  the  fleet  of  fielded 
operational  F-35  aircraft  from  achieving  planned, 
Service-funded  flying  hour  goals.  Original  Service 
bed-down  plans  were  based  on  F-35  squadrons  ramping 
up  to  a  steady  state,  fixed  number  of  flight  hours  per  tail 
per  month,  allowing  for  the  projection  of  total  fleet  flight 
hours. 

-  Since  poor  availability  in  the  field  has  shown  that  these 
original  plans  were  unexecutable,  the  Program  Office  has 
since  produced  “modeled  achievable”  projections  of  total 
fleet  flight  hours,  basing  these  projections  on  demonstrated 
fleet  reliability  and  maintainability  data,  as  well  as 
expectations  for  future  improvements.  The  most  current 
modeled  achievable  projection  is  from  November  2014. 

■  Through  November  23,  2015,  the  fleet  had  flown 
approximately  82  percent  of  the  modeled  achievable 
hours.  This  is  an  improvement  since  October  2014,  the 
date  used  in  the  FY 14  DOT&E  Annual  Report,  when  the 
fleet  had  flown  only  72  percent  of  modeled  achievable 
hours,  but  it  is  still  below  expectation. 

■  The  F-35B  variant  has  flown  approximately  11  percent 
more  hours  than  its  modeled  achievable  projection, 

in  part  due  to  a  ramped  up  level  of  flying  to  produce 
trained  pilots  for  the  Marine  Corps  IOC  declaration. 

•  The  following  table  shows  by  variant  the  planned  versus 
achieved  flight  hours  for  both  the  original  plans  and  the 
modeled-achievable  for  the  fielded  production  aircraft 
through  November  23,  2015. 


F-35  FLEET  PLANNED  VS.  ACHIEVED  FLIGHT  HOURS  AS  OF  NOVEMBER  23, 201 5 

Variant 

Original  Bed-Down  Plan 
Cumulative  Flight  Hours 

"Modeled  Achievable" 
Cumulative  Flight  Hours 

Estimated 

Planned 

Achieved 

Percent 

Planned 

Estimated 

Planned 

Achieved 

Percent 

Planned 

F-35A 

26,000 

16,768 

65% 

22,000 

16,768 

76% 

F-35B 

14,000 

12,156 

87% 

11,000 

12,156 

111% 

F-35C 

5,500 

2,949 

54% 

6,000 

2,949 

49% 

Total 

45,500 

31,873 

70% 

39,000 

31,873 

82% 
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F-35  Fleet  Reliability 

•  Aircraft  reliability  assessments  include  a  variety  of  metrics, 
each  characterizing  a  unique  aspect  of  overall  weapon 
system  reliability. 

-  Mean  Flight  Hours  Between  Critical  Failure  (MFHBCF) 
includes  all  failures  that  render  the  aircraft  not  safe  to 
fly,  and  any  equipment  failures  that  would  prevent  the 
completion  of  a  defined  F-35  mission.  It  includes  failures 
discovered  in  the  air  and  on  the  ground. 

-  Mean  Flight  Hours  Between  Removal  (MFHBR)  gives 
an  indication  of  the  degree  of  necessary  logistical  support 
and  is  frequently  used  in  determining  associated  costs. 

It  includes  any  removal  of  an  item  from  the  aircraft  for 
replacement  with  a  new  item  from  the  supply  chain.  Not 
all  removals  are  failures,  and  some  failures  can  be  fixed 
on  the  aircraft  without  a  removal.  For  example,  some 
removed  items  are  later  determined  to  have  not  failed 
when  tested  at  the  repair  site.  Other  components  can  be 
removed  due  to  excessive  signs  of  wear  before  a  failure, 
such  as  worn  tires. 

-  Mean  Flight  Hours  Between  Maintenance  Event 
Unscheduled  (MFHBME  Unsch)  is  a  useful  reliability 
metric  for  evaluating  maintenance  workload  due  to 
unplanned  maintenance.  Maintenance  events  are  either 
scheduled  (e.g.,  inspections,  planned  removals  for  part 
life)  or  unscheduled  (e.g.  maintenance  to  remedy  failures, 
troubleshooting  false  alarms  from  fault  reporting  or  defects 
reported  but  within  limits,  unplanned  servicing,  removals 
for  worn  parts — such  as  tires).  One  can  also  calculate  the 
mean  flight  hours  between  scheduled  maintenance  events, 
or  total  events  including  both  scheduled  and  unscheduled. 
However,  for  this  report,  all  MFHBME  Unsch  metrics 
refer  to  the  mean  flight  hours  between  unscheduled 
maintenance  events  only,  as  it  is  an  indicator  of  aircraft 
reliability  and  the  only  mean-flight-hour-between- 
maintenance-event  metric  with  an  ORD  requirement. 

-  Mean  Flight  Hours  Between  Failure,  Design  Controllable 
(MFHBFDC)  includes  failures  of  components  due  to 
design  flaws  under  the  purview  of  the  contractor,  such 

as  the  inability  to  withstand  loads  encountered  in  normal 
operation.  Failures  induced  by  improper  maintenance 
practices  are  not  included. 

•  The  F-35  program  developed  reliability  growth  projections 
for  each  variant  throughout  the  development  period  as  a 
function  of  accumulated  flight  hours.  These  projections  are 
shown  as  growth  curves,  and  were  established  to  compare 
observed  reliability  with  target  numbers  to  meet  the 
threshold  requirement  at  maturity,  defined  by  75,000  flight 
hours  for  the  F-35A  and  F-35B,  and  by  50,000  flight  hours 
for  the  F-35C,  and  200,000  cumulative  fleet  flight  hours. 

In  November  2013,  the  program  discontinued  reporting 
against  these  curves  for  all  ORD  reliability  metrics,  and 
retained  only  the  curve  for  MFHBF  DC,  which  is  the  only 
reliability  metric  included  in  the  JSF  Contract  Specification 
(JCS).  DOT&E  reconstructed  the  growth  curves  for  the 
other  metrics  analytically  for  this  report  and  shows  them  in 


the  tables  on  the  following  page  for  comparison  to  achieved 
values. 

•  As  of  late  November  2015,  the  F-35,  including  operational 
and  flight  test  aircraft,  had  accumulated  approximately 
43,400  flight  hours,  or  slightly  below  22  percent  of  the  total 
200,000-hour  maturity  mark  defined  in  the  ORD.  Unlike  the 
following  table,  which  accounts  only  for  fielded  production 
aircraft,  the  flight  test  aircraft  are  included  in  the  fleet  hours 
which  count  toward  reliability  growth  and  maturity.  By 
variant,  the  F-35  A  had  flown  approximately  22,300  hours,  or 
30  percent  of  its  individual  75,000-hour  maturity  mark;  the 
F-35B  had  flown  approximately  15,800  hours,  or  21  percent 
of  its  maturity  mark;  and  the  F-35C  had  flown  approximately 
5,300  hours,  or  11  percent  of  its  maturity  mark. 

•  The  program  reports  reliability  and  maintainability  metrics  on 
a  three-month  rolling  window  basis.  This  means  for  example, 
the  MFHBR  rate  published  for  a  month  accounts  only  for  the 
removals  and  flight  hours  of  that  month  and  the  two  previous 
months.  This  rolling  three-month  window  provides  enough 
time  to  dampen  out  variability  often  seen  in  month-to-month 
reports,  while  providing  a  short  enough  period  to  distinguish 
current  trends. 

•  The  first  table  on  the  following  page  compares  current 
observed  and  projected  interim  goal  MFHBCF  values, 
with  associated  flight  hours.  It  shows  the  ORD  threshold 
requirement  at  maturity  and  the  values  in  the  FY14  DOT&E 
Annual  Report  for  reference  as  well. 

•  The  following  similar  tables  compare  current  observed  and 
projected  interim  goals  for  MFHBR,  MFHBME  Unsch,  and 
MFHBF  DC  rates  for  all  three  variants.  MFHBF  DC  is 
contract  specification,  and  its  JCS  requirement  is  shown  in  lieu 
of  an  ORD  threshold. 

•  Note  that  more  current  data  than  May  2015  are  not  available 
due  to  the  Lockheed  Martin  database  (FRACAS)  not  being 
compliant  with  all  applicable  DOD  information  assurance 
policies  mandated  by  U.S.  Cyber  Command. 

•  Reliability  values  increased  for  11  of  12  metrics  between 
August  2014  and  May  2015.  The  only  metric  which  decreased 
in  value  was  MFHBCF  for  the  F-35C.  A  more  in-depth  trend 
analysis  shows,  however,  that  MFHBCF  for  the  F-35C  is 
likely  increasing  over  time,  albeit  erratically.  The  MFHBCF 
metric  shows  particularly  high  month-to-month  variability 

for  all  variants  relative  to  the  other  metrics,  due  to  the  smaller 
number  of  reliability  events  that  are  critical  failures.  For  the 
F-35C  in  particular,  the  August  2014  value  was  well  above 
average,  considering  the  preceding  and  following  months, 
while  the  May  2015  value  was  below  average  for  the  past  year. 

•  Despite  improvements  over  the  last  year,  8  of  the  12  reliability 
metrics  are  still  below  interim  goals,  based  on  their  reliability 
growth  curves,  to  meet  threshold  values  by  maturity.  Two 

of  these  eight  metrics  however,  are  within  5  percent  of  their 
goal,  F-35B  MFHBCF  and  F-35C  MFHBME  Unsch.  The 
remaining  four  are  above  their  growth  curve  interim  values. 

Of  the  four  metrics  above  their  growth  curve  interim  values, 
three  are  the  contract  specification  metric  MFHBF  DC  for 
each  variant;  and  for  this  specific  metric,  the  program  is 
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reporting  F-35B  and  F-35C  reliability  currently  at  or  above 
the  threshold  at  maturity.  The  fourth  metric  that  is  above 
the  growth  curve  interim  value  is  F-35B  MFF1BME  Unsch. 
This  is  the  only  one  of  nine  ORD  metrics  that  is  above  its 
interim  growth  curve  value.  This  pattern  indicates  that, 
although  reliability  is  improving,  increases  in  the  contract 
specification  reliability  metric  are  not  translating  into 
equally  large  improvements  in  the  other  reliability  metrics, 
which  are  operational  requirements  that  will  be  evaluated 
during  IOT&E. 

The  F-35B  is  closest  to  achieving  reliability  goals,  while 
the  F-35A  is  furthest.  For  the  F-35B,  two  of  four  reliability 
metrics  are  above  their  growth  curves,  one  is  within 
5  percent,  and  one  is 
below,  MFHBR.  MFHBR 
is  the  only  metric  where 
all  three  variants  are 
less  than  95  percent  of 
their  interim  goal.  For 
the  F-35A  and  F-35C, 
the  only  metrics  above 
their  growth  goals  are 
the  contract  specification 
metrics,  MFHBFDC. 

One  of  three  F-35C  ORD 
metrics  is  within  5  percent 
of  its  growth  goal,  and 
all  remaining  F-35A  and 
F-35C  ORD  metrics 
are  below  their  interim 
targets  for  this  stage  of 
development. 

The  effect  of  lower 
MFF1BCF  values  is 
reduced  aircraft  full 
mission  capability,  mission 
capability,  and  availability 
rates.  MFF1BR  values 
lagging  behind  their 
growth  targets  drive  a 
higher  demand  for  spare 
parts  from  the  supply 
system  than  originally 
envisioned.  When 
MFF1BME  Unsch  values 
are  below  expectation, 
there  is  a  higher  demand 
for  maintenance  manpower 
than  anticipated. 

DOT&E  updated 
an  in-depth  study  of 
reliability  growth  in 
MFHBR  and  MFHBME 
Unsch  provided  in  the 
FY 14  DOT&E  Annual 
Report.  The  original  study 


examined  the  period  from  July  2012  through  October  2013, 
and  modeled  reliability  growth  using  the  Duane  Postulate, 
which  characterizes  growth  by  a  single  parametric  growth 
rate.  Mathematically,  the  Duane  Postulate  assesses  growth 
rate  as  the  slope  of  the  best-fit  line  when  the  natural 
logarithm  of  the  cumulative  failure  rate  is  plotted  against  the 
natural  logarithm  of  cumulative  flight  hours.  A  growth  rate 
of  zero  would  indicate  no  growth,  and  a  growth  rate  of  1.0  is 
the  theoretical  upper  limit,  indicating  instantaneous  growth 
from  a  system  that  exhibits  some  failures  to  a  system  that 
never  fails.  The  closer  the  growth  rate  is  to  1.0  the  faster 
the  growth,  but  the  relationship  between  assessed  growth 
rates  is  not  linear,  due  to  the  logarithmic  nature  of  the  plot. 


F-35  RELIABILITY:  MFHBCF  (HOURS) 

Variant 

ORD  Threshold 

Values  as  of  May  31 , 201 5 

Values  as  of  August  2014 

Flight 

Hours 

MFHBCF 

Cumulative 

Flight 

Hours 

Interim  Goal 
to  Meet  ORD 
Threshold 
MFHBCF 

Observed 
MFHBCF 
(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Goal 

Cumulative 

Flight 

Hours 

Observed 
MFHBCF 
(3  Mos.  Rolling 
Window) 

F-35A 

75,000 

20 

15,845 

16.1 

10.2 

63% 

8,834 

8.2 

F-35B 

75,000 

12 

11,089 

9.2 

8.7 

95% 

7,039 

7.5 

F-35C 

50,000 

14 

3,835 

10.0 

7.4 

74% 

2,046 

8.3 

F-35  RELIABILITY:  MFHBR  (HOURS) 

Variant 

ORD  Threshold 

Values  as  of  May  31 , 201 5 

Values  as  of  August  2014 

Flight 

Hours 

MFHBR 

Cumulative 

Flight 

Hours 

Interim  Goal 
to  Meet  ORD 
Threshold 
MFHBR 

Observed 
MFHBR 
(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Goal 

Cumulative 

Flight 

Hours 

Observed 
MFHBR 
(3  Mos.  Rolling 
Window) 

F-35A 

75,000 

6.5 

15,845 

5.3 

4.7 

89% 

8,834 

3.1 

F-35B 

75,000 

6.0 

11,089 

4.6 

3.9 

85% 

7,039 

2.5 

F-35C 

50,000 

6.0 

3,835 

4.3 

3.4 

79% 

2,046 

2.3 

F-35  RELIABILITY:  MFHBME  Unsch  (HOURS) 

Variant 

ORD  Threshold 

Values  as  of  May  31 , 201 5 

Values  as  of  August  2014 

Flight 

Hours 

MFHBME 

Unsch 

Cumulative 

Flight 

Hours 

Interim  Goal 
to  Meet  ORD 
Threshold 
MFHBME 
Unsch 

Observed 

MFHBME 

Unsch 

(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Goal 

Cumulative 

Flight 

Hours 

Observed 

MFHBME 

Unsch 

(3  Mos.  Rolling 
Window) 

F-35A 

75,000 

2.0 

15,845 

1.60 

1.18 

74% 

8,834 

0.85 

F-35B 

75,000 

1.5 

11,089 

1.15 

1.32 

115% 

7,039 

0.96 

F-35C 

50,000 

1.5 

3,835 

1.02 

1.00 

98% 

2,046 

0.84 

F-35  RELIABILITY:  MFHBFJDC  (HOURS) 

Variant 

JCS  Requirement 

Values  as  of  May  31 , 201 5 

Values  as  of  August  2014 

Flight 

Hours 

MFHBF_ 

DC 

Cumulative 

Flight 

Hours 

Interim  Goal 
to  Meet  JCS 
Requirement 
MFHBF_DC 

Observed 
MFHBF_DC 
(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Goal 

Cumulative 

Flight 

Hours 

Observed 
MFHBF_DC 
(3  Mos.  Rolling 
Window) 

F-35A 

75,000 

6.0 

1 5,845 

4.6 

4.8 

104% 

8,834 

4.0 

F-35B 

75,000 

4.0 

11,089 

2.9 

4.3 

148% 

7,039 

3.5 

F-35C 

50,000 

4.0 

3,835 

2.6 

4.0 

1 54% 

2,046 

3.6 
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For  example,  a  growth  rate  of  0.4  would  indicate  reliability 
growth  much  higher  than  twice  as  fast  as  a  growth  rate  of  0.2. 
The  updated  analysis  extended  the  period  examined  from 
July  2012  through  May  2015.  The  analysis  investigated  only 
the  F-35A  and  F-35B  variants  due  to  the  still  low  number  of 
flight  hours  on  the  F-35C.  The  study  evaluated  the  current 
growth  rate,  then,  using  that  rate,  projected  the  reliability 
metric  to  the  value  expected  at  maturity. 

The  study  also  evaluated  the  growth  rate  needed  to  meet  the 
ORD  threshold  value  at  maturity  from  the  current  observed 
value  of  the  reliability  metric.  The  first  table  below  shows 
the  results  of  this  updated  study,  along  with  the  growth  rates 
determined  through  October  2013  from  the  original  study  for 
comparison. 

The  currently  exhibited  growth  rates  for  three  of  the  evaluated 
metrics  are  faster  than  the  growth  rates  exhibited  through 
October  2013. 

The  growth 
rate  for  F-35A 
MFHBME 
Unsch  reduced 
slightly.  For 
both  F-35A 
metrics  and 
for  F-35B 
MFHBR,  the 
growth  rate 
is  still  too 

low  to  meet  the  ORD  threshold  by  maturity.  The  analyses 
project  that  if  the  current  growth  rate  holds  constant,  the 
F-35A  MFFIBR  metric  will  achieve  within  90  percent  of  its 
requirement,  while  F-35B  MFFIBME  Unsch  will  significantly 
exceed  its  requirement.  DOT&E  does  not  expect  the  F-35B 
MFFIBME  Unsch  growth  to  sustain  its  current  rate  out 
through  75,000  flight  hours,  but  there  is  plenty  of  margin  for 
the  rate  to  drop  and  still  exceed  the  requirement  by  maturity. 
The  above  growth  rates  were  calculated  with  around  16,000 
hours  for  the 
F-35A,  and  11,000 
hours  for  the  F-35B. 

For  comparison, 
observed 
MFHBME  Unsch 
growth  rates  for 
several  historical 
aircraft  are  shown 
in  the  table  to  the 
right. 

These  growth 

rates  can  still  change,  either  increase  or  decrease,  as  the 
program  introduces  more  reliability  improvement  initiatives 
and  depending  on  how  well  they  pan  out  in  the  field.  Also, 
the  Block  2B  release  expanded  the  aircraft’s  flight  envelope 
and  delivered  initial  combat  capabilities.  As  a  result,  the 
fielded  units  will  likely  fly  their  aircraft  more  aggressively 
to  the  expanded  envelope,  and  use  mission  systems  more 
heavily  than  in  the  past.  This  change  in  operational  use  may 


uncover  new  failure  modes  that  have  an  impact  on  sustaining 
or  increasing  reliability  growth  rates.  Note  that  the  above 
analysis  covers  a  time  span  preceding  Block  2B  fleet  release. 
The  growth  rates  that  the  F-35  must  achieve  and  sustain 
through  75,000  flight  hours,  in  order  to  comply  with  ORD 
performance  thresholds  by  maturity,  have  been  demonstrated 
in  the  past,  but  mostly  on  bombers  and  transports.  The  F-22 
achieved  a  MFHBME  Unsch  growth  rate  of  0.22,  slightly  less 
than  the  slowest  growth  rate  the  F-35  must  sustain,  for  F-35  A 
MFHBR,  and  only  with  an  extensive  and  dedicated  reliability 
improvement  program. 

A  number  of  components  have  demonstrated  reliability  much 
lower  than  predicted  by  engineering  analysis.  This  drives 
down  the  overall  system  reliability  and  can  lead  to  long 
wait-times  for  re-supply  as  the  field  demands  more  spare  parts 
than  the  program  planned  to  provide.  Aircraft  availability  is 

also  negatively 
affected  by  longer- 
than-predicted 
component  repair 
times.  The  table 
below,  grouped 
by  components 
common  to 
all  variants, 
shows  some  of 
the  high-driver 
components 

affecting  low  availability  and  reliability,  followed  by 
components  failing  more  frequently  on  a  particular  variant  or 
which  are  completely  unique  to  it. 


HIGH  DRIVER  COMPONENTS  AFFECTING  LOW  AVAILABILITY  AND 
RELIABILITY 

Common  to  All  Variants 

Additional  High  Drivers 
by  Variant 

F-35A 

•  Avionics  Processors 

•  Nutplate  and  Engine  Heat  Blanket 
Cure  Parameters 

•  Low  Observable  Maintenance 

•  Main  Landing  GearTires 

•  Fuel  System  Components  (Pumps 
and  Valves) 

•  Exhaust  Nozzle 
Converging-Diverging  Link 

•  Data  Transfer  Cartridge 

F-35B 

•  Upper  Lift  Fan  Door  Actuator1 

•  Flexible  Linear  Shaped  Charge 

F-35C 

•  Lightning  Strike  Damage 

•  Nose  Landing  Gear  Launch  Bar 
Bolt2 

1.  Unique  to  the  F-35B. 

2.  Unique  to  the  F-35C. 

Maintainability 

•  The  amount  of  time  needed  to  repair  aircraft  to  return  them 
to  flying  status  remains  higher  than  the  requirement  for 
the  system  when  mature,  but  has  improved  over  the  past 
year.  The  program  assesses  this  time  with  several  measures, 
including  Mean  Corrective  Maintenance  Time  for  Critical 
Failure  (MCMTCF)  and  Mean  Time  To  Repair  (MTTR)  for 
all  unscheduled  maintenance.  MCMTCF  measures  active 


Metric 

Variant 

May 

2015 

Value 

Current 
Growth  Rate 
from  Duane 
Postulate 

Projected 
Value  at 
75,000 
Flight 
Hours 

ORD 

Threshold 

Projected 
Value  as 
%  ORD 
Threshold 

October 
2013 
Growth 
Rate  from 
Duane 
Postulate 

Growth 

Rate 

Needed  to 
Meet  ORD 

MFHBR 

F-35A 

4.7 

0.204 

6.0 

6.5 

93% 

0.129 

0.228 

F-35B 

3.9 

0.243 

4.8 

6.0 

81% 

0.210 

0.297 

MFHBME 

F-35A 

1.18 

0.142 

1.34 

2.0 

67% 

0.162 

0.281 

F-35B 

1.32 

0.427 

2.74 

1.5 

183% 

0.347 

0.244 

Aircraft 

MFHBME 
Growth  Rate 

F-15 

0.14 

F-16 

0.14 

F-22  (at  35,000  flight  hours) 

0.22 

B-1 

0.13 

"Early"  B-2  (at  5,000  flight  hours) 

0.24 

"Late"B-2 

0.13 

C-1 7  (at  1 5,000  flight  hours) 

0.35 
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maintenance  time  to  correct  only  the  subset  of  failures 
that  prevent  the  F-35  from  being  able  to  perform  a  specific 
mission,  and  indicates  how  long  it  takes,  on  average,  for 
maintainers  to  return  an  aircraft  to  Mission  Capable  status. 
MTTR  measures  the  average  active  maintenance  time  for  all 
unscheduled  maintenance  actions,  and  is  a  general  indicator 
of  the  ease  and  timeliness  of  repair.  Both  measures  include 
active  touch  labor  time  and  cure  times  for  coatings,  sealants, 
paints,  etc.,  but  do  not  include  logistics  delay  times  such  as 
how  long  it  takes  to  receive  shipment  of  a  replacement  part. 

•  The  tables  below  compare  measured  MCMTCF  and  MTTR 
values  for  the  three-month  period  ending  in  May  2015  to 
the  ORD  threshold  and  the  percentage  of  the  value  to  the 
threshold  for  all  three  variants.  The  tables  also  show  the 
value  reported  in  the  FY14  DOT&E  Annual  Report  for 
reference.  For  all  variants,  the  MCMTCF  and  MTTR  times 
decreased  (improved),  with  particularly  strong  decreases 
for  the  F-35  A  and  F-35B  MCMTCF.  The  F-35  A  improved 
to  a  much  larger  degree  than  either  the  F-35B  or  F-35C. 
Nonetheless,  both  maintainability  measures  for  all  variants 
were  well  above  (worse  than)  the  ORD  threshold  value 
required  at  maturity.  Note  that  more  current  data  than 
May  2015  are  not  available  due  to  the  Lockheed  Martin 
database  (FRACAS)  not  being  compliant  with  all  applicable 
DOD  information  assurance  policies  mandated  by  U.S. 
Cyber  Command. 


F-35  MAINTAINABILITY:  MCMTCF  (HOURS) 

Variant 

ORD 

Threshold 

Values  as  of 
May  31, 201 5 
(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Threshold 

Values  as  of 
August  2014 
(3  Mos.  Rolling 
Window) 

F-35A 

4.0 

9.7 

243% 

15.6 

F-35B 

4.5 

10.2 

227% 

15.2 

F-35C 

4.0 

9.6 

240% 

11.2 

F-35  MAINTAINABILITY:  MTTR  (HOURS) 

Variant 

ORD 

Threshold 

Values  as  of 
May  31, 201 5 
(3  Mos.  Rolling 
Window) 

Observed 
Value  as 
Percent  of 
Threshold 

Values  as  of 
August  2014 
(3  Mos.  Rolling 
Window) 

F-35A 

2.5 

4.9 

196% 

8.6 

F-35B 

3.0 

7.1 

237% 

7.5 

F-35C 

2.5 

5.8 

232% 

6.6 

•  More  in-depth  analysis  between  May  2014  and  May  2015, 
in  order  to  capture  longer-term  one-year  trends,  shows  that 
MCMTCF  and  MTTR  for  all  three  variants  are  decreasing 
(improving),  but  with  high  month-to-month  variability.  For 
MCMTCF,  the  rate  of  decrease  for  the  F-35  A  and  F-35B  is 
the  highest,  while  improvements  for  the  F-35C  have  been 
slower  to  manifest.  For  MTTR,  the  rate  of  improvement 
has  been  greatest  for  the  F-35  A,  and  slightly  slower  for  the 
F-35B  and  F-35C. 

•  Several  factors  contribute  to  lengthy  maintenance  durations, 
especially  adhesive  cure  times  for  structural  purposes, 


such  as  attaching  hardware  (e.g.,  nutplates  and  installing 
heat  blankets  around  the  engine),  as  well  as  long  material 
cure  times  for  low  observable  repairs.  From  July  2014 
to  June  2015,  program  records  show  that  maintenance  on 
“attaching  hardware,”  such  as  nutplates  and  heat  blankets, 
absorbed  approximately  20  percent  of  all  unscheduled 
maintenance  time,  while  low  observable  repairs  accounted 
for  15  percent;  these  were  the  two  highest  drivers.  The 
increased  use  of  accelerated  curing  procedures,  such  as 
blowing  hot  air  on  structural  adhesives  or  low  observable 
repair  pastes  to  force  a  quicker  cure,  may  account  for  some 
of  the  decrease  in  repair  times  over  the  past  year,  but  much 
room  remains  for  improvement.  The  third  highest  driver 
of  unscheduled  maintenance,  work  on  the  ejection  seat,  by 
contrast,  only  accounted  for  3  percent  of  all  unscheduled 
maintenance  hours. 

•  The  immature  state  of  the  maintenance  manuals  and 
technical  information  maintainers  use  to  fix  aircraft  may 
also  negatively  affect  long  repair  times.  The  program  is 
still  in  the  process  of  writing  and  verifying  Joint  Technical 
Data  (JTD)  (see  separate  section  in  this  report).  Whenever 
maintainers  discover  a  problem  with  no  solution  yet  in 
JTD,  and  this  problem  prevents  the  aircraft  from  flying,  the 
maintainers  must  submit  a  “Category  I”  Action  Request 
(AR)  to  a  joint  government/Lockheed  Martin  team  asking 
for  tailored  instructions  to  fix  the  discrepancy.  This  team 
can  take  anywhere  from  several  days  to  nearly  a  month 

to  provide  a  final  response  to  each  AR,  depending  on 
the  severity  and  complexity  of  the  issue.  The  number  of 
final  Category  I  AR  responses  per  aircraft  per  month  has 
been  slowly  increasing  from  December  2014  through 
August  2015.  This  trend  indicates  that,  as  the  fleet 
matures,  maintainers  are  continuing  to  face  failure  modes 
not  adequately  addressed  by  the  JTD  or  that  require  new 
repair  instructions.  However,  there  are  other  reasons  for 
submitting  an  AR,  which  may  also  partly  account  for  this 
increasing  trend.  For  example,  depot  teams  submit  ARs  for 
depot-related  repair  work.  More  aircraft  cycling  through 
the  modifications  program,  therefore,  drives  some  of  this 
increase.  In  addition,  supply  occasionally  delivers  parts  with 
missing,  incomplete,  or  incorrect  electronic  records,  known 
as  Electronic  Equipment  Logs  (EELs),  preventing  those  parts 
from  being  incorporated  into  the  aircraft’s  overall  record  in 
Autonomic  Logistics  Information  System  (ALIS).  In  these 
cases,  squadron  maintenance  personnel  cannot  electronically 
certify  the  aircraft  safe  for  flight  until  supply  delivers  correct 
EELs,  and  maintenance  personnel  submit  an  AR  to  request 
these  EELs. 

•  A  learning  curve  effect  is  also  likely  improving  repair  times. 
As  maintainers  become  more  familiar  with  common  failure 
modes,  their  ability  to  repair  them  more  quickly  improves 
over  time. 

•  Maintainers  must  dedicate  a  significant  portion  of  F-35 
elapsed  maintenance  time  to  scheduled  maintenance 
activities  as  well,  which  also  affects  aircraft  availability 
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rates  in  addition  to  repair  times.  Scheduled  maintenance 
accounted  for  55  percent  of  all  maintenance  time  from 
June  2014  to  July  2015.  (Scheduled  maintenance  time  does 
not  appear  in  either  the  MCMTCF  or  MTTR  metrics.) 

•  Reducing  the  burden  of  scheduled  maintenance  by  increasing 
the  amount  of  time  between  planned  in-depth  and  lengthy 
inspections  that  are  more  intrusive  than  routine  daily 
inspections  and  servicing,  will  have  a  positive  effect  on 
how  often  aircraft  are  available  to  fly  missions,  provided 
experience  from  the  field  warrants  such  increases.  An 
example  is  the  engine  borescope  inspection,  which  were 
required  after  the  engine  failure  on  AF-27  in  June  2014.  The 
interval  for  these  inspections  increased  after  the  program 
determined  a  fix  to  the  cause  of  the  failure  and  began 
implementing  it  on  fielded  aircraft,  it  will  take  more  time 
and  experience  with  field  operations  to  collect  data  that 
show  whether  the  program  can  increase  inspection  intervals 
without  affecting  aircraft  safety  for  flight  though. 

Autonomic  Logistics  Information  System  (ALIS) 

•  The  program  develops  and  fields  the  ALIS  in  increments, 
similar  to  the  method  for  fielding  mission  systems  capability 
in  the  air  vehicle.  In  2015,  the  program  fielded  new  versions 
of  both  hardware  and  software  to  meet  requirements  for 

the  Marine  Corps  IOC.  Although  the  program  adjusted 
both  schedule  and  incremental  development  build  plans  for 
ALIS  hardware  and  software  multiple  times  in  2014,  it  held 
the  schedule  more  stable  in  2015  by  deferring  capabilities 
to  later  software  versions.  The  Program  Office  released 
several  new  versions  of  the  software  used  in  ALIS  in  2015. 
Flowever,  each  new  version  of  software,  while  adding 
some  new  capability,  failed  to  resolve  all  the  deficiencies 
identified  in  earlier  releases.  Throughout  2015,  formal 
testing  of  ALIS  software  has  taken  place  at  the  Edwards  AFB 
flight  test  center  on  non-operationally  representative  ALIS 
hardware,  which  relies  on  reach-back  capability  to  the  prime 
contractor  at  Fort  Worth.  The  program  still  does  not  have  a 
dedicated  end-to-end  developmental  testing  venue  for  ALIS, 
but  has  begun  plans  to  develop  one  at  Edwards  AFB.  This 
test  venue,  referred  to  as  the  Operationally  Representative 
Environment  (ORE),  will  operate  in  parallel  with  the 
ALIS  squadron  unit  assigned  to  the  operational  test 
squadrons.  The  program  plans  to  have  the  ORE  in  place 
as  early  as  spring  2016.  The  ORE  is  planned  to  be  a 
replicate  of  a  full  ALIS  system  and  is  needed  to  complete 
developmental  testing  of  ALIS  hardware  and  software  in  a 
closed  environment  to  manage  discoveries  and  corrections 
to  deficiencies  prior  to  OT&E  and  fielding  to  operational 
units.  Meanwhile,  formal  testing,  designated  as  Logistics 
Test  and  Evaluation  (LT&E),  remains  limited  and  differs 
from  how  field  units  employ  ALIS.  For  example,  the  flight 
test  center  at  Edwards  AFB  does  not  use  Prognostic  Health 
Management  (PHM),  Squadron  Health  Management  (SHM), 
Anomaly  and  Failure  Resolution  System  (AFRS),  and  the 
Computerized  Maintenance  Management  System  (CMMS), 


each  of  which  are  modules  within  ALIS  that  the  operational 
units  use  routinely. 

ALIS  Software  Testing  and  Fielding  in  20 1 5 

•  During  2015,  the  program  accomplished  the  following  with 
ALIS  software: 

-  The  program  transitioned  all  fielded  units  from  ALIS  1.0.3 
to  2.0.0  between  January  and  April  2015.  This  software 
includes  integrated  exceedance  management,  improved 
interfaces  with  legacy  government  systems,  an  upgrade 
to  Microsoft  Windows  7  on  laptop  and  other  portable 
devices,  fixes  to  deficiencies,  and  reduced  screen  refresh 
and  download  times.  Testing  of  software  2.0.0  identified 
two  Category  1  deficiencies  (same  categorization  as 
previously  explained  in  this  report  in  “Mission  Systems” 
section),  both  of  which  remained  uncorrected  when  the 
program  delivered  the  software  to  field  units.  According 
to  the  program’s  LT&E  report  on  ALIS  2.0.0,  the  test  team 
identified  the  following  deficiencies: 

■  A  deficiency  in  the  air  vehicle’s  maintenance  vehicle 
interface  (MVI) — the  hardware  used  to  upload  aircraft 
data  files — corrupted  the  aircraft  software  files  during 
the  upload  process.  Technical  manuals  in  ALIS  direct 
the  process  for  loading  aircraft  files.  The  contractor 
addressed  this  deficiency  by  creating  a  fix  in  the  final 
Block  2B  aircraft  software,  and  the  program  fielded  it  in 
2015. 

■  The  Mission  Capability  Override  (MCO)  feature  gives 
maintenance  supervisors  the  authority  and  ability  to 
override  an  erroneous  mission  capability  status  in  ALIS. 
The  LT&E  of  ALIS  1.0.3,  conducted  in  September  and 
October  2012,  revealed  a  discrepancy  in  the  mission 
capability  status  between  two  modules  of  ALIS.  The 
Computerized  Maintenance  Management  System 
(CMMS),  which  uses  Health  Reporting  Codes  (HRCs) 
downloaded  from  the  aircraft,  can  report  an  aircraft  as 
Mission  Capable.  Meanwhile,  another  module  within 
ALIS,  the  Squadron  Health  Management  (SHM),  which 
makes  the  mission  capable  determination  based  on 

the  Mission  Essential  Function  List,  could  categorize 
the  aircraft  as  Non-Mission  Capable  (NMC).  This 
discrepancy  is  a  result  of  errors  in  the  interfaces  between 
HRCs  and  the  list  of  mission  essential  functions.  When 
this  discrepancy  occurs,  maintenance  supervisors 
should  be  able  to  use  the  MCO  feature  to  override  either 
status  within  ALIS,  which  makes  the  aircraft  available 
for  flight.  However,  the  Mission  Capability  Override 
is  deficient  because  it  does  not  allow  override  of  the 
status  within  SHM  (the  override  functions  properly  for 
CMMS).  In  ALIS  2.0.0,  the  same  deficiency  remains. 
However,  ALIS  2.0.0  adds  capabilities  using  the  aircraft 
status  in  SHM  to  collect  the  mission  capable  status  of 
aircraft  across  the  fleet.  Using  SHM  status  to  generate 
fleet  availability  metrics  may  be  inaccurate  because  of 
the  MCO  deficiency. 
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■  In  addition  to  the  Category  1  deficiencies  listed  above, 
the  LT&E  test  team  also  identified  56  Category  2 
deficiencies  (same  categorization  as  previously 
explained  in  this  report  in  “Mission  Systems”  section) 
in  the  AL1S  2.0.0  report.  The  following  list  highlights 
deficiencies,  either  singly  or  in  related  groups,  which 
affect  aircraft  maintenance  and  sortie  generation  rates: 

»  Parts  management  functionality  within  CMMS,  which 
alerts  ALIS  users  if  maintainers  attempt  to  install 
an  incorrect  part  on  an  aircraft  after  the  aircraft  has 
undergone  modification  (i.e.,  modifications  needed 
due  to  concurrency  of  development  with  production), 
is  deficient.  Once  an  aircraft  has  undergone 
modification,  maintainers  should  install  only  specific, 
newer  types  and  models  of  parts.  However,  CMMS 
incorrectly  authorizes  older/inappropriate  replacement 
parts,  changing  the  aircraft  to  an  unauthorized 
configuration,  which  lacks  the  attributes  of  the 
modification.  The  configuration  management  function 
of  CMMS  is  also  deficient,  as  it  does  not  maintain 
accurate  configuration  records  of  aircraft  with 
completed  modifications  when  CMMS  has  permitted 
the  installation  of  infidel  parts  on  the  aircraft. 

»  Maintainers  must  use  manual  workarounds  to  ensure 
the  aircraft  mission  capable  status  is  accurate  if  they 
determine  additional  maintenance  is  required  beyond 
that  dictated  by  the  HRCs  from  the  post-mission 
download.  For  example,  if  maintenance  personnel 
find  or  cause  additional  problems  while  performing 
maintenance,  they  must  create  new  work  orders  with 
appropriate  severity  codes  indicating  that  an  aircraft 
is  no  longer  mission  capable.  However,  CMMS  and 
SHM  will  not  reflect  that  new  aircraft  status,  requiring 
a  maintenance  supervisor  to  open  each  work  order  to 
review  the  actual,  current  aircraft  status. 

»  The  heavy  maintenance  workload,  required  to  enter 
pertinent  maintenance  data  into  ALIS,  causes  field 
units  to  create  workarounds,  including  creating  task 
templates  outside  of  ALIS  to  get  maintenance  records 
into  ALIS. 

»  AFRS,  designed  to  provide  a  library  of  possible 
maintenance  actions  for  each  HRC  does  not  have 
the  troubleshooting  solutions  for  approximately 
45  percent  of  the  HRCs. 

»  Data  products  that  ALIS  is  dependent  on  to  make 
mission  capable  determinations,  such  as  HRCs, 
the  HRC  nuisance  filter  list,  AFRS  troubleshooting 
libraries,  and  the  mission  essential  function  list,  do 
not  sufficiently  manage  configuration  by  including 
version,  release  date,  applicability,  or  record  of 
changes.  As  a  result,  maintenance  personnel  spend 
additional  time  correlating  the  data  files  to  the 
individual  aircraft — a  process  which  increases  the 
risks  of  errors  and  loss  of  configuration  management 
of  the  aircraft  assigned  to  the  units. 


»  Long  wait  times  to  synchronize  the  Portable 
Maintenance  Aid  to  transfer  work  order  data  to  the 
ALIS  squadron  unit. 

»  Long  wait  times  needed  to  complete  data  searches, 
export  reports,  and  apply  processes  within  ALIS. 

-  The  program  developed  ALIS  2.0.1  to  upgrade  to 
Windows  Server  12,  add  new  capabilities  required  to 
support  the  Marine  Corps’  IOC  declaration  in  mid-2015, 
and  address  ALIS  2.0.0  deficiencies.  The  program 
completed  the  LT&E  of  ALIS  2.0.1  in  May  2015,  but 
results  were  poor,  so  the  program  did  not  release  the 
software  to  the  field.  As  of  the  writing  of  this  report,  the 
program  had  not  signed  out  the  ALIS  2.0.1  LT&E  report. 
According  to  their  “quick  look”  briefing,  the  test  team 
discovered  five  new  Category  1  deficiencies  and  confirmed 
that  the  contractor  did  not  correct  in  ALIS  2.0.1  the  two 
Category  1  deficiencies  identified  during  ALIS  2.0.0 
testing  (listed  above).  According  to  the  briefing,  the 

five  new  Category  1  deficiencies  are: 

■  The  Integrated  Exceedance  Management  System, 
designed  to  assess  and  report  whether  the  aircraft 
exceeded  limitations  during  flight,  failed  to  function 
properly.  The  Services  require  proper  functioning  of  this 
capability  to  support  post-flight  maintenance/inspections 
and  safe  turnaround  for  subsequent  flights. 

■  AFRS,  which  is  critical  to  troubleshooting  and 
maintenance  repairs,  demonstrated  unstable  behavior 
and  frequently  failed  because  of  interface  problems  and 
a  system  licensing  configuration  issue. 

■  ALIS  randomly  prevented  user  logins. 

■  The  maintenance  action  severity  code  functionality 
in  CMMS — designed  to  automatically  assign  severity 
codes  to  work  orders  as  maintenance  personnel  create 
them — did  not  work  correctly. 

■  ALIS  failed  to  process  HRCs  correctly  when 
maintenance  personnel  used  CD  media  to  input 
them  into  ALIS  at  sites  that  do  not  use  PMD  readers 
(described  below)  to  download  maintenance  data. 

-  The  program  developed  another  version  of  ALIS,  version 
2.0. 1.1,  which  contained  numerous  software  “patches” 
designed  to  correct  the  five  Category  1  deficiencies 
discovered  by  the  test  team  during  the  LT&E  of 

ALIS  2.0.1.  The  test  team  conducted  an  LT&E  in  May  and 
June  2015  specifically  to  determine  if  Lockheed  Martin 
had  resolved  each  deficiency.  The  test  team  evaluated  the 
correction  for  each  deficiency  as  the  contractor  delivered 
the  software  patches.  As  of  the  end  of  November, 
the  program  had  not  signed  out  the  LT&E  report  on 
ALIS  2.0. 1.1,  but  according  to  the  test  team’s  “quick  look” 
briefing,  they  recommended  releasing  ALIS  2. 0.1.1  to 
the  fielded  units,  which  the  program  completed  between 
July  and  October  2015.  In  their  “quick  look”  briefing, 
the  test  team  also  noted  failures  of  redundant  systems 
and  workarounds  that  were  required  to  address  other 
unresolved  problems.  These  included: 
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■  Frequent  failures  of  the  aircraft  memory  device,  which 
serves  as  a  back  up  to  the  PMD,  to  download  into  ALIS 
when  the  PMD  is  corrupted. 

■  CMMS  and  SFIM  exhibited  disparities  in  tracking 
on-aircraft  equipment  usage  which  required  maintainers 
to  develop  and  operate  a  parallel  tracking  system 
independent  of  ALIS. 

■  Managing  data  loads  associated  with  mission 
planning  required  extensive  contractor  support  as  the 
maintenance-vehicle  interface  did  not  support  direct 
loading  to  the  aircraft  as  designed. 

■  Air  vehicle  data  transfer  between  squadron  hardware, 
required  for  deployments  and  aircraft  induction  to  and 
from  depots,  required  extensive  contractor  support. 

■  Air  vehicle  lockdown  capability,  needed  for  impounding 
an  aircraft  in  the  event  of  an  investigation,  did  not  work. 

-  All  versions  of  ALIS  have  demonstrated  persistent 
problems  with  data  quality  and  integrity,  particularly  in 
the  Electronic  Equipment  Logbooks  (EELs),  which  allow 
usage  tracking  of  aircraft  parts.  Frequently,  EELs  are  not 
generated  correctly  or  do  not  transfer  accurately,  requiring 
manual  workarounds  that  extend  aircraft  repair  and 
maintenance  times.  Without  accurate  EELs  data,  ALIS  can 
improperly  ground  an  aircraft  or  permit  an  aircraft  to  fly 
when  it  should  not. 

ALIS  Hardware  Fielding  in  20 1 5 

•  During  CY15,  the  program  demonstrated  progress  in  the 
development  and  fielding  of  ALIS  hardware  and  aligning 
hardware  versions  with  the  software  versions  noted  above. 

-  The  program  delivered  the  first  deployable  version  of 
the  Squadron  Operating  Unit  (SOU),  deemed  SOU 
V2  (Increment  1),  aligned  with  ALIS  software  2.0.1, 
to  MCAS  Yuma  to  support  Marine  Corps  IOC.  The 
originally  fielded  unit-level  hardware,  SOU  VI,  failed 
to  meet  ORD  deployability  requirements  due  to  its  size 
and  weight.  SOU  V2  incorporates  modular  components 
that  meet  two-man-carry  transportation  requirements  and 
decrease  set-up  time.  Additionally,  field  units  can  tailor 
the  SOU  V2  by  adjusting  the  number  of  components  with 
which  they  deploy  depending  on  projected  duration.  SOU 
V2  allowed  the  program  to  meet  requirements  for  Marine 
Cops  IOC.  It  will  support  Block  2B,  3i,  and  3F  aircraft. 
The  program  plans  to  field  one  set  of  SOU  V2  hardware 
for  each  F-35  unit  and  an  additional  set  of  SOU  hardware 
for  each  F-35  operating  location.  During  partial  squadron 
deployments,  the  unit  will  deploy  with  their  SOU  V2  while 
the  remainder  of  the  squadron’s  aircraft  will  transfer  to  the 
base-level  SOU. 

-  Because  the  Edwards  AFB  flight  test  center  does  not  have 
an  SOU  V2,  the  program  conducted  the  hardware  portion 
of  the  LT&E  at  Fort  Worth  in  May  2015.  Testing  included 
demonstrating  that  PMDs  from  aircraft  at  the  flight  test 
center  downloaded  correctly  into  the  SOU  V2. 

-  The  program  continued  to  field  PMD  readers  to  operating 
locations.  As  designed,  maintainers  download  aircraft 


PMDs  post  flight  to  ALIS  through  a  Ground  Data  Security 
Assembly  Receptacle  (GDR).  However,  it  takes  between 
1.0  and  1.2  hours  to  download  all  data  from  a  1-hour  flight. 
PMD  readers  download  maintenance  data  only  within 
5  minutes,  permitting  faster  servicing  of  aircraft. 

-  The  program  delivered  an  SOU  V2  to  the  JOTT  at 
Edwards  AFB  in  November  2015.  This  SOU  V2  will  be 
“on  loan”  from  Hill  AFB,  Utah,  and  is  planned  to  be  used 
in  an  F-35A  deployment  to  Mountain  Home  AFB,  Idaho, 
in  March  2016  with  six  Air  Force  F-35A  aircraft. 

-  Lockheed  Martin  delivered  full  SOU  V2  kits  to  MCAS 
Yuma  in  May  2015  and  to  the  Pilot  Training  Center  at 
Luke  AFB,  Arizona  (for  Norway)  in  October.  Because 
Israel  did  not  require  an  SOU  V2  when  scheduled  for 
delivery,  the  Program  Office  arranged  for  it  to  go  to 
MCAS  Yuma  in  November  2015,  so  the  squadron  could 
use  it  in  an  assessment  of  the  F-35B’s  capabilities  at  an 
austere  location.  The  program  delivered  an  SOU  V2 
deployment  kit  to  Nellis  AFB  and  a  Central  Point  of  Entry 
(CPE)  kit,  which  included  a  CPE  and  an  SOU  VI,  for 
United  Kingdom  lab  use,  in  December  2015.  A  full  SOU 
kit  includes  more  peripheral  equipment  than  a  deployment 
kit. 

Cross  Ramp  Deployment  Demonstration  May  201 5 

•  During  April  and  May  2015,  the  Air  Force’s  Air  Combat 
Command  tasked  the  31st  Test  and  Evaluation  Squadron 
(TES)  at  Edwards  AFB  to  conduct  a  limited  deployment  of 
F-35A  aircraft  as  part  of  the  de-scoped  Block  2B  operational 
test  activity.  This  deployment,  from  one  hangar  on  the 
flight  line  at  Edwards  AFB  to  another  hangar,  termed  the 
Cross  Ramp  Deployment  Demonstration  (CRDD),  gave  the 
program  and  the  Air  Force  an  opportunity  to  learn  how  to 
deploy  the  F-35  air  system  and  ALIS.  Originally,  the  31st 
TES  planned  to  use  ALIS  2.0.1,  but  delays  in  releasing  that 
software  resulted  in  the  need  to  use  ALIS  2.0.0  instead. 
Overall,  the  CRDD  showed  that  ALIS  2.0.0  deficiencies,  plus 
difficulties  encountered  during  the  CRDD  in  downloading 
and  transferring  data  files  from  home  station  to  a  deployed 
location,  will  negatively  affect  sortie  generation  rate  if 
they  remain  uncorrected.  The  CRDD  also  demonstrated 
that  getting  ALIS  2.0.0  online  with  current  maintenance 
infonnation  while  also  conducting  flying  operations  is  time 
consuming,  complex,  and  labor  intensive.  Working  around 
ALIS  2.0.0  deficiencies  in  this  manner  was  possible  for  this 
demonstration  of  limited  duration;  however,  it  would  not  be 
acceptable  for  deployed  combat  operations. 

-  The  3 1st  TES  deployed  across  the  ramp  on  the  flightline 
by  packing  and  moving  an  ALIS  SOU  VI  loaded  with 
ALIS  2.0.0  software,  mission  planning  hardware, 
maintenance  personnel,  support  equipment,  and  tools. 
Three  F-35A  aircraft  “deployed”  to  the  cross  ramp  location 
after  the  ALIS  SOU  VI  was  in  place.  For  supply  support, 
maintenance  personnel  obtained  spare  parts  from  the 
base  warehouse  as  though  they  had  not  deployed  (i.e., 

the  31st  TES  did  not  deploy  in  this  demonstration  with  a 
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pre-planned  set  of  spares  as  an  operational  unit  would  have 
for  an  actual  deployment). 

Transfer  of  aircraft  data  from  the  SOU  at  the  main 
operating  location  to  the  SOU  at  the  “deployed”  location 
and  getting  the  SOU  online  took  several  days  to  complete 
and  required  extensive  support  from  Lockheed  Martin 
ALIS  administrators,  a  level  of  effort  not  planned  for  the 
deployment  and  not  operationally  suitable.  Although 
not  finalized  by  the  Services,  deployment  concepts  of 
operation  will  include  procedures  for  transferring  aircraft 
data  between  SOUs  via  secure  electronic  methods.  The 
test  team  attempted  the  primary  electronic  method,  but 
the  configuration  of  the  deployed  SOU  caused  it  to  fail. 
Ultimately,  data  transfer  occurred  using  the  physical 
transfer  of  back-up  CDs  to  the  deployed  location,  but  the 
3 1st  TES  could  not  load  the  files  until  the  end  of  the  third 
of  the  five  days  of  flight  operations,  because  administrators 
had  to  load  multiple  software  patches,  and  resolve  ALIS 
account  problems  for  every  authorized  user.  After  loading 
the  aircraft  data  on  the  deployed  SOU,  administrators 
also  had  to  enter  manually  all  maintenance  performed  on 
the  aircraft  during  this  time  into  the  SOU  before  bringing 
ALIS  online  to  support  operations. 

Flight  operations  did  take  place  without  the  support  of 
normal  ALIS  operations  for  the  three  days  while  the  test 
team  worked  to  get  the  SOU  online.  During  this  period, 
maintenance  personnel  prepared  and  recovered  aircraft 
without  a  full  post-mission  download  of  maintenance  data, 
including  health  and  fault  codes  normally  captured  and 
transmitted  to  ALIS  2.0.0.  The  deployed  aircraft  generally 
required  only  routine  maintenance  such  as  tire  changes, 
which  maintainers  could  complete  without  access  to  all 
maintenance  instructions.  One  aircraft  experienced  a 
radio  failure,  which  did  not  require  an  HRC  download  to 
diagnose,  and  did  not  fly  again  until  maintainers  replaced 
the  radio. 

To  prepare  for  the  deployment,  the  3 1st  TES  did  not  fly 
the  aircraft  designated  for  the  deployment  during  the 
week  prior,  allowing  maintenance  personnel  to  prepare 
the  aircraft  and  ensure  all  inspections  were  current  and 
maintenance  actions  complete.  This  preparation  allowed 
the  unit  to  conduct  flight  operations  for  three  days  during 
the  deployment  while  the  SOU  remained  offline. 

At  the  end  of  the  demonstration,  the  3 1st  TES  successfully 
transferred  data  to  the  Autonomic  Logistics  Operating 
Unit  at  Fort  Worth — per  one  of  the  electronic  methods  of 
transfer  expected  for  deployed  operations — but  staffing 
levels  at  Lockheed  Martin  were  insufficient  to  complete 
the  transfer  all  the  way  back  to  the  home  station  SOU. 
Instead,  the  31st  TES  transferred  data  back  to  the  home 
station  SOU  via  an  alternative,  web-based,  secure,  online 
file  transfer  service  operated  by  the  Army  Missile  Research 
and  Development  Center,  referred  to  as  “AMRDEC.” 

The  CRDD  showed  that  although  cumbersome,  field 
units  could  relocate  the  SOU  VI  hardware  to  a  deployed 
operating  location  and  eventually  support  operations  with 


that  hardware.  However,  difficulties  in  transferring  data 
between  home  station  and  a  deployed  SOU  made  the 
deployment  and  redeployment  processes  time  consuming 
and  required  extensive  support  from  the  contractor  to 
complete.  Although  ALIS  2. 0.1.1  added  improvements 
to  data  transfer  capabilities,  the  program  has  not  yet 
demonstrated  those  improvements  in  a  Service-led 
deployment  exercise.  Therefore,  it  is  unknown  the  extent 
to  which  ALIS  2. 0.1.1  improves  data  transfer  capabilities. 

Marine  Corps  Austere  Assessment  Deployment  Demonstration, 

December  201 5 

•  The  Marine  Corps  deployed  eight  production  F-35B 

aircraft — six  from  VMFA-121  at  Marine  Corps  Air  Station 
(MCAS)  Yuma,  Arizona,  and  two  from  VMX-22  at  Edwards 
AFB,  California — to  the  Strategic  Expeditionary  Landing 
Field  (SELF)  near  MCAS  Twentynine  Palms,  California, 
from  December  8  -  15,  2015,  to  assess  deployed  operations 
to  an  austere,  forward-base  location.  The  Marine  Corps 
aligned  the  deployment  with  a  combined  arms  live  fire 
exercise,  Exercise  Steel  Knight,  to  have  the  F-35  detachment 
provide  close  air  support  for  the  rest  of  the  exercise 
participants  as  the  forward  deployed  air  combat  element 
(ACE).  The  SELF  had  an  airfield  constructed  of  AM2 
matting  (aluminum  paneling  engineered  for  rapid  runway 
construction  to  support  austere  operations)  and  minimal 
support  infrastructure,  which  required  the  Marine  Corps 
to  deploy  the  necessary  support  equipment,  spare  parts, 
and  personnel;  and  set  up  secure  facilities  on  the  flightline 
to  conduct  F-35B  flight  operations.  Although  it  was  not 
a  formal  operational  test  event,  the  JOTT  and  DOT&E 
staff  observed  operations  and  collected  data  to  support  the 
assessment. 

-  While  deployed,  and  in  support  of  the  exercise,  the  Marine 
Corps  flew  approximately  46  percent  of  the  planned 
sorties  (28  sorties  flown  versus  61  sorties  planned), 

not  including  the  deployment,  redeployment,  and  local 
familiarization  sorties.  Accounting  for  all  sorties  (i.e., 
deploying  and  redeploying,  local  training,  aircraft  diverts 
and  swapping  one  aircraft  at  home  station)  the  Marine 
Corps  flew  approximately  54  percent  of  scheduled  sorties 
(82  scheduled  versus  44  flown).  Weather,  particularly 
high  winds,  aircraft  availability,  and  problems  transferring 
aircraft  data  from  the  home  station  to  the  deployed  ALIS 
SOU  all  contributed  to  the  loss  of  scheduled  sorties. 

-  The  Marine  Corps  planned  to  employ  inert  GBU-12  and 
GBU-32  weapons  in  the  CAS  role  during  the  exercise. 

The  Marine  Corps  ordnance  loading  teams  completed 
multiple  GBU-12  and  GBU-32  upload  and  download 
evolutions  at  the  SELF.  However,  pilots  released 
fewer  weapons  than  planned  due  to  weather  and  range 
limitations. 

-  Two  aircraft  experienced  foreign  object  damage  to  their 
engines  from  debris  ingested  during  operations,  grounding 
them  until  the  end  of  the  deployment.  The  engine  damage 
on  both  aircraft  was  not  severe  enough  to  cause  an  engine 
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change,  but  required  a  Pratt  and  Whitney  technician, 
certified  in  blending  out  damage  to  engine  blades,  to 
repair  the  engines  on  both  aircraft  at  Twentynine  Palms  so 
they  could  return  to  flyable  status,  allowing  the  aircraft  to 
return  to  home  station  at  the  end  of  the  deployment.  No 
further  action  was  required  for  the  engine  repairs  on  either 
aircraft.  It  was  still  unknown  at  the  time  of  this  report  how 
these  types  of  engine  repairs  would  be  conducted  during 
deployed  or  combat  operations. 

-  The  deployment  was  the  first  to  use  the  AL1S  Standard 
Operating  Unit  Version  2  (SOU  V2),  which  is  smaller, 
lighter  weight  and  more  modularized  than  Version  1 . 
Although  Marine  Corps  AL1S  personnel  were  able  to 
set  up  the  SOU  V2  (i.e.,  place  and  connect  the  modules 
and  apply  power)  within  a  few  hours  after  arrival,  setting 
up  connectivity  with  the  broader  Autonomic  Logistics 
Global  Support  (ALGS)  function  did  not  occur  for  quite 
some  time.  The  Customer  Relations  Module  (CRM)  of 
ALIS,  used  to  submit  action  requests  to  the  contractor 
for  resolving  maintenance  actions,  operated  only 
intermittently  during  the  deployment. 

-  The  transfer  of  data  from  home  station  to  the  deployed 
ALIS  SOU  took  several  days  to  fully  complete,  a  process 
that  is  not  affected  by  the  version  of  SOU  being  used. 

Since  the  SOU  V2  lacked  connectivity  to  the  Autonomic 
Logistics  Operating  Unit,  which  is  required  for  transferring 
data  via  the  preferred  method  of  keeping  the  data  entirely 
within  the  infrastructure  of  ALIS,  initial  data  transfers  for 
the  six  aircraft  from  MCAS  Yuma  were  AMRDEC.  Files 
were  transferred  to  workstations  at  the  deployed  site  and 
then  loaded  into  ALIS  via  CDs.  The  downloading  of  files 
from  AMRDEC  was  slowed  several  times  when  SATCOM 
connectivity  was  lost  during  the  process.  The  aircraft 
from  Edwards  AFB,  however,  brought  CD’s  with  them  for 
transfer  into  ALIS. 

-  The  deployment  provided  valuable  “lessons  learned”  for 
the  Marine  Corps  as  it  develops  concepts  of  operation  for 
forward  basing  and  austere  operations.  While  the  SOU  V2 
proved  to  be  easier  and  quicker  to  set  up  than  the  SOU  VI, 
transferring  aircraft  data  from  home  station  to  the  deployed 
location  continued  to  be  problematic.  Poor  aircraft 
availability  reduced  the  support  the  F-35B  ACE  was  able 
to  provide  to  the  large  force  exercise. 

ALIS  Software  and  Hardware  Development  Planning  through  the 

EndofSDD 

•  In  CY15,  the  program  continued  to  struggle  with  providing 
the  planned  increments  of  capability  to  support  the  scheduled 
releases  of  ALIS  software  2.0.x  and  3.0.x.  The  program 
approved  changes  to  the  content  of  the  ALIS  developmental 
software  release  plan  in  April  2015  for  ALIS  2.0.1  and 
2.0.2.  To  adhere  to  the  previously  approved  software 
release  schedule  for  ALIS  2.0.1,  the  program  deferred 
several  capabilities,  including  cross-domain  solutions  for 
information  exchange  requirements  and  firewall  protections 
for  low  observable  and  mission  planning  data,  to  a  later  fix 


release.  The  Marine  Corps,  which  required  ALIS  2.0. 1  for 
IOC,  supported  the  Program  Office’s  plan  to  defer  these 
capabilities  until  after  IOC. 

-  These  deferrals  are  in  addition  to  decisions  in  2014 
to  defer  life-limited  parts  management  capabilities  to 
ALIS  2.0.2  and  ALIS  3.0.0. 

-  Although  the  re -plan  included  a  two-month  delay  in  the 
LT&E  dates  for  ALIS  2.0. 1  from  March  to  May  2015, 
the  program  did  not  change  the  initial  fielding  date  of 
July  2015,  the  planned  date  for  Marine  Corps  IOC.  The 
program  also  approved  a  fix  release  of  this  software  to 
follow  almost  immediately. 

-  The  program  had  previously  scheduled  fielding  of  software 
2.0.2,  beginning  in  December  2015,  but  approved  a 
nearly  eight-month  delay  to  late  July  2016.  The  Air 
Force  IOC  requirement  is  for  ALIS  software  2.0.2  to  be 
fielded.  Since  the  Air  Force  also  requires  operationally 
representative  hardware  and  software  90  days  before 
declaring  IOC,  the  delayed  schedule  does  not  support 

the  Air  Force  IOC  objective  date  of  August  2016.  An 
additional  potential  problem  is  that  the  program  currently 
does  not  plan  to  conduct  cybersecurity  penetration  testing 
during  the  development  of  this  ALIS  release  or  any 
future  developmental  releases,  but  will  instead  rely  on 
previous,  albeit  limited,  cybersecurity  test  results.  This 
decision  increases  the  risk  that  the  program  will  not  be 
aware  of  ALIS  vulnerabilities  before  making  fielding 
decisions.  However,  the  JOTT  will  complete  operational 
cybersecurity  testing  of  fielded  ALIS  components. 

-  At  an  April  2015  review,  the  program  projected  initial 
fielding  of  ALIS  3.0.0  in  June  2017  and  indicated 
they  would  propose  combining  ALIS  3.0.0  and  3.0. 1 
(previously  planned  for  December  20 1 7)  into  a  single 
release  in  June  2018.  Should  this  occur,  ALIS  software 
will  not  include  full  life  limited  parts  management,  a 
capability  planned  for  Marine  Corps  IOC,  until  nearly 
three  years  after  Marine  Corps  IOC.  All  fielded  locations 
will  require  high  levels  of  contractor  support  until 

the  program  integrates  life  limited  parts  management 
capability  into  ALIS.  In  November  2015,  the  program 
proposed  changing  the  content  of  ALIS  3.0.0  to  reflect 
service  and  partner  priorities  and  moving  the  fielding  date 
forward  by  approximately  six  months. 

-  The  program  has  deferred  the  PHM  downlink  originally 
planned  for  release  in  ALIS  2.0.0  indefinitely  because  of 
security  concerns. 

•  The  program  plans  the  following  hardware  releases  to  align 
with  software  releases  noted  above: 

-  The  program  plans  SOU  V2  (Increment  2)  to  align  with 
ALIS  2.0.2  and  include  additional  SOU  V2  hardware 
improvements  to  support  Air  Force  IOC,  including 
dynamic  routing  to  deliver  data  via  alternate  network  paths 
and  sub-squadron  reporting  to  allow  deployed  assets  to 
report  back  to  a  parent  SOU. 
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-  The  third  increment  of  SOU  V2  hardware  will  address 
Service  requirements  for  decentralized  maintenance, 
allowing  personnel  to  manage  maintenance  tasks  whether 
or  not  they  connect  their  portable  maintenance  aid  (PMA) 
to  the  main  SOU  (the  PMA  provides  connectivity  between 
maintenance  personnel  and  the  aircraft,  enabling  them  to 
do  maintenance  tasks  on  the  aircraft  by  viewing  technical 
data  and  managing  work  orders  downloaded  from  the 
SOU).  Increment  3  will  also  permit  units  to  conduct 
deployments  without  SOU  hardware,  instead  relying  on 
PMAs.  This  increment  of  SOU  V2  will  align  with  ALIS 
release  3.0.0. 

Prognostic  Health  Management  (PHM)  within  ALIS 

•  The  PHM  system  collects  air  system  performance  data  to 
determine  the  operational  status  of  the  air  vehicle  and,  upon 
reaching 
maturity, 
will  use  data 
collected 
across 
the  F-35 
enterprise 
and  stored 
within  PHM 
to  predict 
maintenance 
requirements 
based  on 
trends.  The 
PHM  system 
provides  the 
capability 
to  diagnose 
and  isolate 
failures,  track 
and  trend 

the  health  and  life  of  components,  and  enable  autonomic 
logistics  using  air  vehicle  HRCs  collected  during  flight  and 
saved  on  aircraft  PMDs.  The  F-35  PHM  system  has  three 
major  components:  fault  and  failure  management  (diagnostic 
capability),  life  and  usage  management  (prognostic 
capability),  and  data  management.  PHM  diagnostic  and  data 
management  capabilities  remain  immature.  The  program 
does  not  plan  to  integrate  prognostic  capabilities  until 
ALIS  2.0.2. 

-  Diagnostic  capability  should  detect  true  faults  within 
the  air  vehicle  and  accurately  isolate  those  faults  to  a 
line-replaceable  component.  However,  to  date,  F-35 
diagnostic  capabilities  continue  to  demonstrate  poor 
accuracy,  low  detection  rates,  and  a  high  false  alarm  rate. 
Although  coverage  of  the  fault  detection  has  grown  as 
the  program  has  fielded  each  block  of  F-35  capability, 
all  metrics  of  performance  remain  well  below  threshold 
requirements.  The  table  above  compares  specific 
diagnostic  measures  from  the  ORD  with  current  values  of 
performance  through  June  2015. 
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-  PHM  affects  nearly  every  on-  and  off-board  system  on  the 
F-35.  It  must  be  highly  integrated  to  function  as  intended 
and  requires  continuous  improvements  for  the  system  to 
mature. 

•  Poor  diagnostic  performance  increases  maintenance 
downtime.  Maintainers  often  conduct  built-in  tests  to  see 
if  the  faidt  codes  detected  by  the  diagnostics  are  true  faults. 
False  failures  (diagnostics  detecting  a  failure  when  one  does 
not  exist)  require  service  personnel  to  conduct  unnecessary 
maintenance  actions  and  often  rely  on  contractor  support 
to  diagnose  system  faults  more  accurately.  These  actions 
increase  maintenance  man-hours  per  flight  hour,  which 
in  turn  can  reduce  aircraft  availability  rates  and  sortie 
generation  rates.  Poor  accuracy  of  diagnostic  tools  can  also 
lead  to  desensitizing  maintenance  personnel  to  actual  faults. 


-  Qualified  maintenance  supervisors  can  cancel  an  HRC 
without  generating  a  work  order  for  maintenance  actions 
if  they  know  that  the  HRC  corresponds  to  a  false  alarm 
not  yet  added  to  the  nuisance  filter  list.  In  this  case,  the 
canceled  HRC  will  not  result  in  the  generation  of  a  work 
order,  and  it  will  not  count  as  a  false  alarm  in  the  metrics 
in  the  above  table.  The  program  does  not  score  an  HRC 
as  a  false  alarm  unless  a  maintainer  signs  off  a  work  order 
indicating  that  the  problem  described  by  the  HRC  did  not 
occur.  Because  PHM  is  immature  and  this  saves  time,  it 
occurs  regularly  at  field  locations  but  artificially  lowers  the 
true  false  alarms  rate  (i.e.,  actual  rate  is  higher). 

-  Comparing  the  values  in  the  table  above  with  previous 
reports,  Mean  Flight  Hours  Between  Flight  Safety  Critical 
False  Alarms  is  the  only  diagnostic  metric  that  has  shown 
significant  improvement  over  the  last  year.  Other  metrics 
have  stayed  either  flat  or  decreased  (worsened)  slightly. 


METRICS  OF  DIAGNOSTIC  CAPABILITY 

(6 -month  rolling  window  as  of  June  2015.  Data  provided  by  the  Program  Office  are  considered  "preliminary"as  they  have  not  completed 

the  formal  adjudication  process  by  the  data  review  board.) 

Diagnostic  Measure 

Threshold 

Requirement 

Demonstrated  Performance 

Block  1 

Block  2 

Block  3 

Developmental  Test  and  Production  Aircraft 

Fault  Detection  Coverage  (percent  mission  critical  failures  detectable  by  PHM) 

N/A 

65 

73 

84 

Fault  Detection  Rate  (percent  correct  detections  for  detectable  failures) 

98 

65 

73 

85 

Fault  Isolation  Rate  (percentage):  Electronic  Fault  to  One  Line  Replaceable 
Component  (LRC) 

90 

68 

69 

72 

Fault  Isolation  Rate  (percentage):  Non-Electronic  Fault  to  One  LRC 

70 

76 

72 

79 

Fault  Isolate  Rate  (percentage):  None-Electronic  Fault  to  3  or  Fewer  LRC 

90 

82 

87 

87 

Production  Aircraft  Only 

Mean  Flight  Hours  Between  False  Alarms 

50 

0.20 

0.60 

0.18 

Mean  Flight  Hours  Between  Flight  Safety  Critical  False  Alarms 

450 

1,360 

543 

170 

Accumulated  Flight  Hours  for  Measures 

N/A 

1,360 

4,886 

1,360 

Ratio  of  False  Alarms  to  Valid  Maintenance  Events 

N/A 

44:1 

16:1 

1079:1 
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-  The  following  lists  the  systems  most  likely  to  result  in 
missed  fault  detections,  incorrect  fault  isolations,  and  false 
alarms  as  of  June  2015: 

■  Missed  detections.  Integrated  Core  Processor,  power 
and  thermal  management  system,  panoramic  color 
display,  communications-navigation-identification  (CNI) 
rack  modules,  and  the  Helmet  Mounted  Display  System. 

■  Incorrect  isolation.  Integrated  Core  Processor,  power 
and  thermal  management  system,  electronic  warfare, 
fuel  system,  CNI  rack  modules,  and  hydraulic  power 
system. 

■  False  alarms.  CNI  system,  propulsion,  electronic 
warfare,  suspension  and  release,  displays  and  indicators 
in  general. 

Off-board  Mission  Support  (OMS)  within  ALIS 

•  OMS  provides  F-35  ground  mission  planning,  mission 
debrief,  security,  and  sensor  management  capabilities. 

Similar  to  other  components  of  ALIS,  the  program  does 
not  have  a  developmental  test  venue  for  OMS.  Mission 
planning  modules  include  the  baseline  Joint  Mission 
Planning  System  software  that  pilots  and  tacticians  use  to 
develop  files  for  uploading  into  the  aircraft  prior  to  flight. 
OMS  includes  separate  hardware  such  as  workstations  and 
encryption/decryption  devices  and  networks  with  ALIS  for 
data  management.  In  addition  to  mission  planning,  OMS 
provides  the  following  functions: 

-  Ground  security  that  allows  for  secure  data  management 
and  cryptologic  key  management  at  multiple  classification 
levels 

-  Sensor  management  and  selection  of  mission  data  files  to 
create  a  mission  data  load 

-  Mission  debrief  capability  for  replaying  audio  and  video 
from  completed  flights 

•  Until  September  2015,  the  training  center  did  not  provide 
hands-on  training  on  OMS,  requiring  the  pilots  to  learn 

it  through  trial  and  error  and  by  asking  questions  of 
the  contractor.  Also,  the  program  has  not  yet  provided 
OMS  user  manuals.  As  a  result,  field  units  will  likely 
have  difficulty  providing  the  expertise  to  create  tailored, 
theater-specific  mission  data  loads  during  contingency 
operations.  Few  pilots  currently  possess  the  training  and 
experience  to  build  mission  data  loads  from  beginning  to 
end. 

•  OMS  deficiencies  will  have  a  negative  impact  on  combat 
mission  and  training  flight  operational  tempo.  Long 
processing  times  create  bottlenecks  in  both  mission  planning 
and  mission  debrief,  particularly  for  multi-ship  missions. 

-  Pilots  transfer  a  mission  plan  into  the  PMD  using  a  GDR, 
which  encrypts  the  information.  The  PMD  loading 
process  is  unnecessarily  complex,  taking  25  to  45  minutes 
to  transfer  a  mission  data  load  from  an  OMS  workstation 
to  a  PMD.  If  pilots  transfer  the  same  mission  data  load 

to  multiple  PMDs  for  a  multi-ship  mission,  each  PMD  is 
encrypted  separately  with  no  time  savings. 


-  OMS  requires  excessive  time  for  loading  of  PMDs  and 
decryption  of  mission  data  and  does  not  support  timely 
mission  debrief,  particularly  if  pilots  fly  multiple  missions 
in  one  day.  For  example,  a  1-hour  mission  typically  takes 
between  1.0  and  1.2  hours  to  decrypt,  and  depending  on 
the  amount  of  cockpit  video  recorded,  can  take  longer. 

-  Administrative  functions  in  OMS,  such  as  theater  data  load 
updates,  user  authentication  file  updates,  cryptographic 
updates,  and  data  transfers  are  inefficient  and  require 
excessive  times  to  complete. 

■  Because  of  cryptographic  key  expirations,  OMS 
administrators  must  update  the  theater  data  load  at  least 
every  28  days.  The  OMS  administrator  builds  the  load 
on  OMS  equipment,  transfers  it  to  a  separate  laptop, 
creates  a  CD,  and  then  uploads  it  to  the  SOU.  Again, 
personnel  cannot  build  cryptographic  key  loads  on 

one  OMS  workstation  and  export  it  to  others  in  the  same 
unit;  they  must  build  them  individually. 

■  Personnel  must  install  cryptographic  keys  on  the  aircraft, 
OMS  workstations,  GDRs,  and  GDR  maintenance 
laptops. 

■  Block  2B  aircraft  have  33  different  cryptographic  keys 
with  varying  expiration  periods.  When  building  a  key 
for  the  entire  jet,  an  error  frequently  means  rebuilding 
from  the  beginning,  which  can  take  several  hours. 

■  The  cryptographic  key  management  tool  is  not  intuitive, 
prone  to  errors,  and  does  not  have  a  validation  function 
so  the  user  can  determine  if  a  key  load  is  accurate  prior 
to  transferring  it  to  the  aircraft. 

■  Loading  of  incorrect  keys  can  result  in  aircrew  not 
having  capabilities  such  as  secure  voice  transmissions. 

■  Local  security  policies  vary,  making  hardware 
requirements  and  information  technology  processes 
different  at  each  operating  location. 

-  Current  OMS  hardware  does  not  have  the  necessary 
video  processing  and  display  capabilities  to  allow  pilots 
to  effectively  debrief  a  multi-ship  mission.  Current 
debriefing  capability  via  laptops  does  not  provide  adequate 
resolution  or  a  large  enough  presentation  for  a  four-ship 
debrief. 

Joint  Technical  Data  (JTD) 

•  Although  the  verification  of  Joint  Technical  Data  (JTD) 
modules  has  proceeded  through  2015,  field  units  continue 
to  face  challenges  where  JTD  is  either  not  yet  verified,  is 
unclear,  or  includes  errors.  To  work  around  these  challenges, 
personnel  must  frequently  submit  ARs  to  the  contractor  and 
wait  for  the  engineering  disposition,  a  process  that  adds  to 
maintenance  time. 

•  The  program  identifies  JTD  modules  and  the  primary 
contractors  develop  and  verify  them  in  the  field.  Once  JTD 
modules  complete  verification,  the  program  includes  them  in 
the  JTD  package  distributed  periodically  to  all  field  locations 
through  ALIS.  At  the  field  locations,  they  are  downloaded  to 
unit-level  SOUs  and  PMAs.  JTD  updates  currently  require 
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downloading  of  the  entire  JTD  package  (i.e.,  partial  changes 
to  JTD  cannot  be  distributed  to  fielded  units). 

-  ALIS  release  2.0.0  included  Trilogi  Viewer  4.0,  which 
supports  delivery  of  partial  builds  and  amendments  to 
JTD  to  reduce  the  time  required  to  install  JTD  updates 
at  the  unit  level.  However,  the  program  determined 
that  this  version  of  Trilogi  contains  a  software  error, 
which  prevented  implementation  of  this  capability  until 
corrected.  As  of  December  2015,  the  program  continues  to 
distribute  only  complete,  bundled  JTD  packages. 

-  The  total  number  of  identified  data  modules  grows 
each  year  as  the  program  matures  and  low-rate  initial 
production  (LRIP)  contracts  include  additional  JTD 
delivery  requirements.  The  air-vehicle  JTD  includes  time 
compliance  technical  data,  depot-level  technical  data, 

air  vehicle  diagnostics  and  troubleshooting  procedures, 
complete  structural  field  repair  series  data,  aircraft  battle 
damage  assessment  and  repair,  and  maintenance  training 
equipment.  According  to  the  most  recent  data  from  the 
Program  Office,  as  of  September  2015,  propulsion  JTD 
development  is  nearly  complete  and  on  schedule.  To 
support  Marine  Corps  IOC,  the  contractor  focused  on 
development  ofF-35B  unit-level 
and  Supportable  Low  Observable 
(SLO)  JTD  and  deferred 
approximately  260  data  modules, 
identified  by  the  Marine  Corps  as 
not  needed  until  after  IOC,  such  as 
JTD  modules  for  loading  weapons 
not  yet  cleared  for  use. 

-  Although  the  program  included 
development  of  support  equipment 
JTD  in  the  SDD  contract,  the 
program  funded  additional 
support  equipment  via  another, 
separate  contract,  which  requires 
approximately  1,700  more  data 
modules.  The  contract  began  in 
July  2014  and  the  modules  must  be 
verified  before  the  end  of  SDD. 

-  The  program  estimates  that 
development  of  all  JTD  for  each 
variant  of  the  air  vehicle  and  for 
propulsion  will  meet  Service 
milestones. 

•  DOT&E  sees  risk  in  the  ability  of 
the  program  to  complete  air  vehicle 
JTD  verifications  in  time  to  meet 
Service  needs.  The  program  does 
not  have  a  formal  JTD  verification  schedule  nor  dedicated 
time  to  complete  air  vehicle  JTD  verifications.  In  addition,  it 
depends  on  the  availability  of  aircraft,  primarily  at  Edwards 
and  Eglin  AFBs,  to  complete  this  work.  JTD  verifications 
have  lower  priority  than  maintaining  the  flight  schedule, 
so  verification  teams  normally  cannot  schedule  dedicated 
events. 


-  The  program  did  focus  on  completing  F-35B  unit-level 
verifications  during  2015  with  verifications  lagging 
development  by  fewer  than  200  modules  out  of 
5,157  developed. 

-  The  program  will  not  complete  highly  invasive  JTD 
verifications,  such  as  those  for  removing  fuel  cells,  until  an 
aircraft  requires  this  level  of  maintenance. 

-  The  program  did  not  fund  SLO  JTD  verifications  until 
March  2014,  so  SLO  JTD  lags  other  verification  efforts. 
However,  most  SLO  JTD  verification  will  take  place  using 
desktop  analysis,  and  the  program  expects  verification  for 
all  variants  to  proceed  on  schedule. 

•  As  of  September  2015,  the  program  had  verified  approximately 
94  percent  of  the  identified  air  vehicle  JTD  modules  for 
the  F-35A,  93  percent  of  the  F-35B,  and  73  percent  of  the 
F-35C.  The  table  below  shows  the  number  of  JTD  modules 
identified,  developed,  and  verified  for  the  air  vehicle  by 
variant,  pilot  flight  equipment,  support  equipment,  and  SLO. 
Overall,  approximately  77  percent  of  these  modules  have 
been  identified,  developed,  and  verified.  The  program  tracks 
propulsion  JTD  separately. 


Air-Ship  Integration  and  Ship  Suitability  Testing 
F-35B 

•  The  Marine  Corps  conducted  a  suitability  demonstration 
with  six  operational  (i.e.,  non-test  fleet)  F-35B  aircraft 
onboard  the  US  S  Wasp  from  May  18  -  29,  2015. 

-  Despite  bearing  the  title  “OT-1”  for  “Operational 
Test  -  One,”  as  expected,  the  demonstration  was  not 


F-35  SDD  JOINTTECHNICAL  DATA  (JTD)  DEVELOPMENT  AND  VERIFICATION  STATUS 
REQUIRED  BY  COMPLETION  OF  SYSTEM  DEVEOLOPMENT  AND  DEMONSTRATION  (SDD)  CONTRACT 
Air  Vehicle,  Pilot  Flight  Equipment  (PFE),  Support  Equipment  (SE),  and  Supportable  Low  Observable  (SLO) 

(as  of  end  of  September  201 5) 


Module 

Type 

Modules 

Identified 

Modules 

Developed 

Percent 

Identified 

Modules 

Developed 

Number  of 
Verification 
Events' 

Percent  Identified 
Modules  Verified 

F-35A2 

Unit-level 

4,603 

4,326 

94% 

4,328 

Not  Determined 

F-35B2 

Unit-level 

5,335 

5,157 

97% 

4,966 

93% 

F-35C2 

Unit-level 

4,766 

4,009 

84% 

3,488 

73% 

Common 
(all  variants)3 

Unit-level 

84 

58 

69% 

62 

Not  Determined 

PFE 

Common 

326 

318 

98% 

274 

84% 

SE 

Common 

2,345 

1,596 

68% 

1,351 

58% 

SLO 

F-35A 

745 

599 

79% 

80 

11  % 

F-35B 

739 

739 

100% 

428 

58% 

F-35C 

668 

97 

15% 

79 

12% 

Common 

6 

6 

100% 

4 

67% 

TOTAL 

19,617 

16,905 

86% 

15,060 

77% 

1 .  For  F-35A  and  Common  modules,  multiple  verifications  are  required  for  some  single  data  modules,  hence  values  represent 

verifications  and  exceed  the  number  of  modules  developed. 
2.  Includes  field-  and  depot-level  JTD  for  operations  and  maintenance,  on-  and  off-equipment  JTD, 

and  structured  field  repairs. 

3.  Includes  aircraft  JTD  for  general  repairs,  sealants,  bonding,  structured  field  repairs,  and 

non-destructive  investigations. 
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an  operational  test  and  could  not  demonstrate  that  the 
F-35B  is  operationally  effective  or  suitable  for  use  in  any 
type  of  limited  combat  situation.  This  was  due  to  many 
factors  concerning  how  the  demonstration  was  structured 
including,  but  not  limited  to,  the  following  major  features 
that  were  not  operationally  representative: 

■  Other  aircraft  of  a  standard  Air  Combat  Element 
(ACE) — with  which  the  F-35B  would  normally 
deploy — were  not  present,  except  for  the  required 
search  and  rescue  helicopters,  granting  the  F-35B  unit 
practically  sole  use  of  the  flight  deck  and  hangar  bay. 

■  The  embarked  F-35B  aircraft  lacked  the  full 
complement  of  electronic  mission  systems  necessary  for 
combat,  and  not  all  the  normal  maintenance  procedures 
necessary  to  keep  those  systems  in  combat-capable  state 
of  readiness  were  exercised. 

■  The  aircraft  did  not  have  the  appropriate  flight 
clearances  to  carry  or  employ  any  ordnance.  Ordnance 
evolutions  were  limited  to  maintainers  uploading  and 
downloading  inert  bombs  and  missiles  on  the  flight 
deck. 

■  Uniformed  maintainers  had  not  yet  been  equipped 
with  complete  maintenance  manuals  and  mature 
troubleshooting  capabilities,  necessitating  the  extensive 
use  of  contractor  maintenance  personnel  that  would  not 
be  present  on  a  combat  deployment. 

■  Production-representative  support  equipment  was  not 
available.  Instead,  the  demonstration  used  interim 
support  equipment  cleared  for  hangar  bay  use  only  and 
requiring  workarounds  for  conducting  maintenance, 
such  as  fueling  operations,  on  the  flight  deck. 

■  The  operational  logistics  support  system,  known  as 
the  Autonomic  Logistics  Global  Sustainment  system, 
was  not  available.  A  potentially  non-representative 
set  of  spare  parts  was  loaded  onboard  the  ship,  and  the 
program  and  Marine  Corps  provided  extensive  supply 
support  to  ensure  replacement  parts  reached  the  ship 
faster  than  would  be  expected  in  deployed  combat 
operations. 

-  The  USS  Wasp  demonstration  event  did,  however,  provide 
useful  training  for  the  Marine  Corps  and  amphibious 
Navy  with  regards  to  F-35B  operations  onboard  L-class 
ships,  and  also  provided  findings  relevant  to  the  eventual 
integration  of  the  F-35B  into  the  shipboard  environment. 

-  The  Marine  Corps  and  Lockheed  Martin  could  not  transfer 
data  for  the  aircraft,  support  equipment,  spare  parts,  and 
personnel  from  ashore  sites  to  the  SOU  onboard  the  ship 
entirely  within  the  AL1S  network  as  originally  envisioned, 
due  to  the  immaturity  of  the  Autonomic  Logistics 
Operating  Unit.  An  attempt  was  made  to  download  the 
data  onto  the  ship  via  other  government  and  contractor 
networks,  but  the  download  rate  over  the  ship’s  network 
proved  too  slow  to  efficiently  move  the  numerous  large 
files.  Finally,  the  data  were  downloaded  off-ship  via 
commercial  Wi-Fi  access,  burned  to  CDs,  and  imported 
directly  onto  the  Wasp’s  SOU.  This  method  of  transferring 


data  would  not  be  acceptable  for  routine  combat 
deployments. 

■  Similarly,  once  the  USS  Wasp  was  underway,  service 
personnel  noted  that  getting  ALlS-related  data  to  the 
ship  to  support  flight  operations,  such  as  the  EEL 
records  for  spare  parts  delivered  by  supply,  was  slow 
over  satellite  communications  channels. 

■  In  addition  to  the  difficulties  moving  the  data  back  and 
forth  between  the  Wasp  SOU  and  ashore  site  SOUs, 
data  discrepancies  were  introduced  during  the  transfer 
process,  including  inconsistencies  and  lost  data. 

Transfer  of  aircraft  data  from  the  shore-based  SOU  to 
the  Wasp  SOU  took  nearly  two  days  to  complete,  and 
maintenance  personnel  were  correcting  discrepancies 
found  in  the  aircraft  data  in  ALIS  for  four  additional 
days.  For  example,  when  the  aircraft  data  files  were 
finally  received  onboard  the  USS  Wasp,  all  outstanding 
parts  requisitions  for  the  aircraft  had  been  stripped. 

The  transfer  of  support  equipment  data  took  10  days  to 
complete  and  maintenance  personnel  were  correcting 
deficiencies  in  the  data  during  the  majority  of  the  at-sea 
period. 

-  Aircraft  reliability  and  maintainability  were  poor  enough 
that  it  was  difficult  for  the  Marines  to  keep  more  than 
two  to  three  of  the  six  embarked  aircraft  in  a  flyable 
status  on  any  given  day,  even  with  significant  contractor 
assistance.  Aircraft  availability  during  the  deployment  was 
approximately  55  percent.  Around  80  percent  availability 
would  be  necessary  to  generate  four-ship  combat 
operations  consistently  with  a  standard  six-ship  F-35B 
detachment. 

-  Aircraft  availability  varied  significantly  from  aircraft  to 
aircraft,  however,  with  some  aircraft  requiring  no  major 
maintenance,  and  other  aircraft  barely  contributing  to 
meaningful  flight  operations.  In  particular,  one  aircraft, 
designation  BF-23,  was  reported  “Full  Mission  Capable 
(FMC)”  for  the  entire  11 -day  duration  of  the  deployment. 
Another  aircraft,  BF-37,  flew  less  than  5  hours, 
including  diverting  to  shore  and  back  for  a  landing  gear 
malfunction,  and  was  not  flyable  for  8  of  the  1 1  days. 
BF-37  was  notable  for  being  in  depot  modification  from 
December  8,  2014,  to  May  8,  2015,  right  before  the 
start  of  the  demonstration.  Fleet  units  have  reported 
initial  reliability  difficulties  with  aircraft  after  they  come 
back  from  long  stays  at  the  depot,  and  the  experience 
with  BF-37  onboard  USS  Wasp  would  support  these 
observations. 

-  Poor  fuel  system  reliability  proved  particularly 
challenging,  in  part  due  to  the  nature  of  the  shipboard 
environment.  The  detachment  experienced  two  major  fuel 
system  failures,  a  fuel  boost  pump  and  a  high  level  float 
valve.  For  fuel  system  maintenance,  the  aircraft  must  be 
drained  of  fuel  and  then  certified  gas-free  of  combustible 
fuel  vapors  before  work  can  proceed.  Onboard  ship,  this 
lengthy  process  must  be  done  in  the  hangar  bay  and  little 
work  on  other  aircraft  in  the  bay  can  occur,  particularly 
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electrical  work  or  hot-work,  due  to  the  risk  of  sparks 
igniting  fuel  vapors.  This  is  less  of  an  issue  on  land,  where 
the  aircraft  can  be  moved  far  away  from  other  aircraft 
while  de-fueling.  The  Marines  decided  to  fly  one  of  these 
aircraft  on  a  one-time  waiver  back  to  shore  and  swap  it 
with  a  replacement  aircraft  in  order  to  keep  flying,  and  not 
over-burden  maintenance.  However,  this  would  not  be 
an  option  when  deployed  in  a  combat  zone.  The  program 
should  increase  fuel  system  reliability,  especially  for  the 
F-35B  and  F-35C  variants. 

The  detachment  staged  all  necessary  personnel,  support 
equipment,  tools,  and  ship’s  facilities  to  conduct  engine 
and  lift-fan  removals  and  installations  in  the  hangar  bay, 
but  did  not  actually  conduct  any,  as  a  basic  fit-check. 

The  amount  of  space  required  for  this  heavy  propulsion 
maintenance  is  substantial  and  could  have  a  significant 
operational  impact  on  ACE  operations  when  far  more 
aircraft  are  present  in  the  hangar  bay  and  on  the  flight 
deck. 

During  the  underway  period,  the  Marines  successfully 
delivered  a  mock  spare  F-35  engine  power  module  to  the 
USS  Wasp  via  internal  carry  on  an  MV-22  tilt-rotor,  and 
returned  it  back  to  shore.  This  concept  demonstration 
opens  up  a  potentially  viable  re-supply  method  for 
the  F-35  engine  power  module,  which  is  too  large  and 
heavy  to  deliver  to  a  ship  at  sea  using  current,  traditional 
replenishment  methods.  Work  remains  to  be  done  to 
ensure  that  this  method  will  not  damage  spare  engine 
modules  but,  if  successful,  will  ease  logistical  support  of 
F-35’s  while  onboard  ship. 

Ordnance  evolutions  included  uploading  and  downloading 
of  inert  AIM- 120  missiles,  and  GBU-12  500-pound  laser 
guided  and  GBU-32  1,000-pound  Global  Positioning 
System-guided  bombs.  In  order  to  load  the  bombs  to  their 
appropriate  stations  in  the  internal  weapons  bay,  the  station 
had  to  be  disconnected  from  the  aircraft,  lowered  and 
coupled  to  the  bomb,  and  then  re-connected  to  the  aircraft 
with  the  bomb  attached.  This  procedure  potentially 
invalidates  pre-ordnance  loading  checks  to  ensure  that  the 
weapons  stations  are  working  properly  (i.e.,  that  they  will 
provide  appropriate  targeting  information  to  the  weapon 
and  release  the  weapon  when  commanded). 

The  lack  of  production-representative  support  equipment 
prevented  the  detachment  from  providing  cooling  air  on 
the  flight  deck,  which  is  necessary  to  prevent  the  avionics 
from  overheating  while  conducting  maintenance  and 
servicing  while  on  external  electrical  power  or  internal 
battery  power.  This  limited  the  ability  to  troubleshoot 
on  the  flight  deck  and  made  refueling  operations  less 
efficient.  The  program  should  demonstrate  regular  flight 
deck  operations  with  the  intended  operational  support 
equipment  before  an  actual  combat  deployment. 

The  program  conducted  several  tests  with  a  Handheld 
Imaging  Tool  (HIT)  that  uses  a  small  radar  to  scan  the 
aircraft  and  determine  its  degree  of  stealth.  The  HIT  can 
be  used  to  scan  for  areas  where  the  Low  Observable  (LO) 


material  needs  to  be  repaired,  as  well  as  to  verify  repairs 
to  LO  materials.  It  is  a  replacement  for  a  previous  Radar 
Verification  Radar,  which  was  too  large  for  efficient  use 
in  the  crowded  hangar  bay  of  an  aircraft  carrier.  Initial 
results  of  the  HIT  testing  looked  very  promising,  although 
further  developmental  work  remains. 

-  Several  other  important  findings  surfaced  from  the 
USS  Wasp  demonstration: 

■  When  the  aircraft  is  on  jacks  in  the  hangar  bay, 
maintainers  must  securely  tie  it  down  to  the  deck  with 
chains  to  ensure  that  the  ship’s  rocking  motion  in  the 
waves  does  not  cause  the  aircraft  to  slip  off.  However, 
the  tie  down  pattern  used  prevented  the  weapons  bay 
doors  from  being  opened  while  the  aircraft  is  on  jacks. 
This  will  prevent  maintainers  from  connecting  cooling 
air,  since  the  intake  port  is  located  in  the  internal 
weapons  bay,  and  may  limit  efficient  completion  of 
landing  gear  maintenance. 

■  With  the  current  software  configuration,  when 
maintainers  apply  external  power  to  the  aircraft,  the 
anti-collision  strobe  lights  come  on  automatically, 
flashing  for  a  few  seconds  until  maintainers  can 
manually  turn  them  off.  This  violates  ship  light’s 
discipline,  and  at  night,  it  can  briefly  blind  flight 
deck  personnel  as  well  as  potentially  reveal  the  ship’s 
position.  The  program  must  change  the  software  to 
prevent  this  occurrence  onboard  ship. 

■  The  L-class  ships  currently  lack  the  facilities  to  analyze 
any  debris  found  on  magnetic  chip  collectors  in  the 
engine  oil  system.  Metal  shavings  in  the  engine  oil 
could  indicate  that  engine  components  such  as  bearings 
may  be  wearing  out,  which  could  cause  the  engine 

to  seize  in  flight.  Currently,  if  maintainers  discover 
chips,  they  will  have  to  down  the  aircraft  and  mail  them 
out  to  a  shore  facility  that  can  analyze  the  shavings 
to  determine  if  the  engine  is  up,  or  requires  particular 
maintenance.  This  process  could  take  several  days. 

■  When  the  aircraft  is  wet  it  is  extremely  slippery.  The 
F-35  sits  higher  off  the  deck  than  legacy  aircraft  so 
falls  off  of  it  can  cause  greater  injury,  or  at  sea,  can 
lead  to  a  man-overboard.  This  is  exacerbated  by  the 
plastic  booties  maintainers  are  supposed  to  wear  when 
working  on  the  aircraft  to  protect  the  LO  coatings. 

The  detachment  decided,  for  safety  reasons,  to  allow 
maintainers  to  work  on  the  aircraft  without  wearing 
these  booties.  The  program  should  investigate  alternate 
footwear  to  continue  to  protect  aircraft  LO  coatings 
while  also  ensuring  the  safety  of  maintainers. 

■  When  aircraft  were  landing  nearby,  the  Maintenance 
Interface  Panel  door  vibrated  alarmingly.  The 
maintainers  have  this  door  open  in  order  to  plug  in 
their  portable  computers  to  get  information  from  the 
aircraft  and  control  it  while  conducting  servicing  and 
maintenance.  The  Marines  resorted  to  assigning  a 
maintainer  to  hold  the  door,  while  another  worked  on 
the  computer  if  flight  operations  were  ongoing  nearby. 
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This  was  an  inefficient  use  of  manpower,  and  the  door 
hinge  should  be  stiffened  to  withstand  the  flight  deck 
environment. 

-  The  Navy  made  several  modifications  to  the  USS  Wasp 
in  order  to  support  F-35B  operations.  The  deployment 
demonstration  provided  the  following  observations  on 
some  of  these  ship  modifications: 

■  Naval  Sea  Systems  Command  installed  a  Lithium-Ion 
battery  charging  and  storage  facility.  The  F-35  relies  on 
270  Volts-Direct-Current  and  28  Volts-Fully-Charged 
Lithium-Ion  batteries,  and  other  assets  that  will  deploy 
onboard  L-class  ships  are  also  predicted  to  make  greater 
use  of  Lithium-Ion  batteries.  However,  Lithium-Ion 
batteries  can  catch  fire  under  certain  circumstances, 
especially  during  charging  and,  due  to  their  chemical 
nature,  cannot  be  extinguished  but  must  burn  themselves 
out.  The  storage  facility  consisted  of  racks  of  lockers 
that  resembled  ovens,  each  with  its  own  exhaust 
system  that  coidd  flue  smoke  and  heat  from  a  battery 
undergoing  “thermal  runaway.”  Battery  charging  would 
occur  only  in  these  lockers.  Despite  a  flaw  relating  to 
the  facility’s  air  conditioning  system  being  installed 
improperly,  the  general  design  appeared  robust  and 
functional. 

■  F-35  pilots  must  conduct  much  of  their  mission  planning 
inside  a  Special  Access  Program  Facility,  a  vault-like 
room  that  is  protected  against  electronic  eavesdropping 
and  is  highly  secure.  The  Navy  installed  a  small  Special 
Access  Program  Facility  to  house  several  classified 
AL1S  components  and  provide  an  area  for  pilot  briefings 
and  debriefings.  This  facility  was  adequate  for  the 
demonstration,  but  was  stretched  to  capacity  to  support  a 
six  F-35B  detachment.  The  Navy  and  Marine  Corps  are 
investigating  concepts  for  equipping  L-class  ships  with 
JSF  “heavy”  ACEs  consisting  of  16  to  20  F-35B’s.  In 
these  cases,  a  much  larger  facility  would  be  required. 

■  The  Navy  applied  a  high-temperature  coating  called 
Thermion  to  the  flight  deck  spots  where  F-35B  aircraft 
will  land,  in  lieu  of  the  traditional  “non-skid”  coating, 
to  withstand  the  F-35B’s  exhaust,  which  is  hotter  than 
the  AV-8B.  One  week  into  flight  operations,  personnel 
noted  several  chips  of  the  first  of  two  layers  of  Thermion 
were  missing  along  a  weld  seam  and  started  monitoring 
the  site  after  each  landing.  No  further  degradation  of 
the  Thermion  was  noted  for  the  rest  of  the  detachment. 
Naval  Sea  Systems  Command  is  analyzing  the 
performance  of  the  coating. 

F-35C 

•  The  second  phase  of  ship  suitability  testing — DT-2 — was 
conducted  from  October  2  -  10,  2015.  Ship  availability 
delayed  the  start  of  DT-2  from  the  planned  date  in 
August  2015.  The  principal  goal  of  DT-2  was  to  perform 
launch  and  recovery  of  the  F-35C  with  internal  stores  loaded. 

-  The  F-35C  sea  trials  are  a  series  of  developmental 
tests  conducted  by  the  program  with  the  principal  goal 
of  supporting  development  of  the  aircraft  launch  and 


recovery  bulletins,  and  the  general  goal  of  characterizing 
ship  suitability  for  operating  and  maintaining  F-35C  on 
a  CVN-class  ship.  During  DT-2,  only  developmental 
test  aircraft  CF-3  and  CF-5,  transient  aircraft  needed 
for  logistical  support,  and  search  and  rescue  helicopters 
deployed  to  the  carrier.  No  air  wing  was  present.  The 
major  contractor  was  responsible  for  maintenance.  ALIS 
was  not  installed  on  the  carrier;  it  was  accessed  via 
satellite  link  to  a  location  ashore. 

-  Testers  accomplished  100  percent  of  threshold  and 
objective  test  points  (280  total  test  points)  over  the  course 
of  17  flights  totaling  26.5  flight  hours.  The  results  of  the 
test  are  still  in  analysis.  In  addition  to  the  principal  goal, 
the  test  points  addressed: 

■  Minimum  end  airspeed  for  limited  afterburner  and 
military  power  catapult  launches.  For  catapult  launches 
that  use  afterburner,  engine  power  is  initially  limited 

to  less  than  full  afterburner  power  while  the  aircraft  is 
static  in  the  catapidt,  but  then  automatically  goes  to  full 
afterburner  power  once  released.  This  power  limitation 
was  in  place  to  reduce  thermal  loads  on  the  Jet  Blast 
Deflectors  (JBDs)  behind  the  aircraft. 

■  Crosswinds  catapults 

■  Recovery  in  high  headwinds 

■  Initial  Joint  Precision  Approach  and  Landing  System 
testing 

■  Qualities  of  the  Gen  III  HMDS  at  night 

■  Running  the  Integrated  Power  Package  (IPP)  and  engine 
in  the  hangar  bay 

■  Engine  and  power  module  logistics  in  the  hangar  bay 

-  There  were  7  bolters  (failure  to  catch  an  arresting  wire)  in 
66  arrestments  during  DT-2.  During  DT-1  (Developmental 
Test  -  One),  there  were  no  unplanned  bolters  in 

122  arrestments.  The  higher  rate  was  expected  since  the 
carrier  arresting  gear  was  not  fully  operational  during 
DT-2.  The  third  arresting  wire  (i.e.,  the  wire  typically 
targeted  in  carrier  landings),  was  removed  from  service 
during  the  test  because  of  a  malfunction. 

-  Testers  ran  the  aircraft’s  IPP,  a  miniature  gas  turbine 
engine  that  can  provide  ground  power,  in  the  hangar  bay. 
They  then  performed  a  low-thrust  engine  run  as  well.  This 
process  simulated  maintenance  checkout  procedures  that 
frequently  occur  in  the  hangar  bay  with  legacy  aircraft. 
During  these  evolutions,  crew  position  the  aircraft  with 

its  tail  pointing  out  of  one  of  the  set  of  hangar  bay  doors 
to  the  aircraft  elevators  to  direct  heat  and  exhaust  away 
from  the  inside  of  the  ship.  For  the  F-35C,  the  unique 
concern  is  that  the  IPP  exhaust  vents  up  towards  the 
hangar  bay  ceiling.  The  test  team  monitored  the  IPP 
exhaust  gas  temperature  to  ensure  it  would  not  damage  the 
ceiling  of  the  hangar  bay.  During  both  the  IPP  run  and  the 
engine-turn,  this  temperature  remained  within  safe  limits. 
Testers  also  collected  noise  data;  analysis  is  ongoing.  The 
team  did  not  collect  data  on  the  potential  build-up  of  IPP 
exhaust  gases  within  the  hangar  bay  atmosphere,  but  plans 
to  collect  these  data  during  DT-3. 
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DT-3,  the  third  and  final  set  of  sea  trials,  will  expand  the 
carrier  operating  envelope  further,  including  external  stores, 
and  is  scheduled  to  occur  in  August  2016. 

The  Navy  is  working  on  the  following  air-ship  integration 
issues,  primarily  for  carriers.  Some  of  the  following  issues 
also  apply  to  F-35B  operations  on  L-class  ships: 

-  Flight  deck  JBDs  may  require  additional  side  panel 
cooling  in  order  to  withstand  regular,  cyclic  limited 
afterburner  launches  from  an  F-35C.  JBDs  are  retractable 
panels  that  re-direct  hot  engine  exhaust  up  and  away 
from  the  rest  of  the  flight  deck  when  an  aircraft  is  at 
high  thrust  for  take-off.  Even  with  this  additional 
cooling,  however,  JBDs  may  be  restricted  in  how  many 
consecutive  F-35C  limited  afterburner  launches  they  can 
withstand  before  they  will  require  a  cool  down  period, 
which  could  affect  the  launch  of  large  “alpha  strikes” 
that  involve  every  aircraft  in  the  air  wing,  a  combat  tactic 
the  Navy  has  used  frequently  in  past  conflicts.  F-35C 
limited-afterburner  launches  are  required  when  the  F-35C 
is  loaded  with  external  weaponry  and  in  a  heavy,  high-drag 
configuration.  The  Navy  estimates  that  an  F-35C  will  have 
3,000  catapult  launches  over  a  35  year  expected  lifespan, 
but  that  no  more  than  10  percent  of  these  launches  will  be 
limited-afterburner. 

-  The  Navy  continues  to  investigate  the  replacement  of  a 
mobile  Material  Handling  Equipment  crane  for  several 
purposes  onboard  carriers,  including,  and  perhaps  most 
importantly,  facilitating  F-35  engine  module  maintenance. 
In  order  to  transfer  spare  F-35  engine  modules  from  their 
containers  onto  a  transportation  trailer,  so  they  can  later 
be  installed  in  an  aircraft,  or  to  take  broken  modules  from 
a  trailer  and  put  them  into  a  shipping  container  to  send 
back  to  an  ashore  repair  site,  a  heavy  lift  capable  crane 

is  required.  Onboard  L-class  ships,  the  Navy  will  use  an 
overhead  bridge  crane  built  into  the  hangar  bay  ceiling,  but 
CVNs  do  not  have  any  similar  ship’s  facility  and  the  Navy 
intends  to  use  a  mobile  crane.  However,  efforts  to  acquire 
a  suitable  crane  have  gone  more  slowly  than  originally 
expected.  Given  procurement  timelines,  the  Navy  must 
proceed  without  any  further  delays  in  order  to  have  an 
appropriate  crane  onboard  ship  in  time  for  the  projected 
first  deployment  of  an  F-35C. 

-  Work  continues  on  developing  triple  hearing  protection 
for  flight  deck  crews,  but  with  little  update  since  the 
FY14  DOT&E  Annual  Report.  Both  the  F-35C  and 
F/A-18E/F  produce  around  149  decibels  of  noise  where 
personnel  are  normally  located  when  at  maximum  thrust 
during  launch  evolutions.  The  Navy  has  determined  that 
53  decibels  of  attenuation  will  be  required  from  a  triple 
hearing  protection  system  to  allow  these  personnel  to  be 
on  deck  for  38  minutes,  or  the  equivalent  of  60  launch 
and  recovery  cycles.  Current  designs  only  achieve  in  the 
mid-40s  decibel  range  of  attenuation,  which  allows  less 
than  10  minutes  of  exposure  before  certain  flight  deck 
personnel  reach  their  maximum  daily  limit  of  noise. 


-  Two  methods  of  shipboard  aircraft  firefighting  for  the  F-35 
with  ordnance  in  the  weapons  bay  are  being  developed, 
one  for  doors  open  and  one  for  doors  closed.  Each  will 
consist  of  an  adapter  that  can  fit  to  the  nozzle  of  a  standard 
hose.  The  open  door  adapter  will  also  attach  to  a  24-foot 
aircraft  tow  bar  so  firefighters  can  slide  it  underneath  the 
aircraft  and  spray  cooling  water  up  into  the  bay. 

■  The  Navy  has  produced  four  articles  of  the  open  bay 
firefighting  device.  This  adapter  performed  well  in 
preliminary  tests  conducted  in  2014.  Three  of  the 
production  articles  have  been  sent  to  Naval  Air  Station 
China  Lake  for  further  evaluation,  and  the  fourth  has 
been  sent  to  a  training  command  at  Naval  Air  Station 
Norfolk  to  begin  training  flight  deck  personnel  in  its  use 
and  ship’s  company  personnel  how  to  maintain  it. 

■  Developmental  work  continues  on  the  closed  bay 
adapter.  The  Navy  is  currently  pursuing  two  different 
designs  that  would  cut  through  the  aircraft  skin  to  flood 
the  weapons  bay  with  water  as  well  as  lock  into  place  to 
allow  firefighting  crews  to  back  away  from  the  fire  after 
the  hose  is  securely  attached  to  the  aircraft.  One  design 
will  require  two  sailors  to  use,  and  the  other  design  is 
more  aggressive,  but  would  potentially  only  require  a 
single  sailor. 

Climatic  Lab  Testing 

•  The  program  conducted  climatic  testing  on  an  F-35B  test 
aircraft  (BF-5)  in  the  McKinley  Climatic  Laboratory  from 
October  2014  to  March  2015.  All  the  planned  environments 
were  tested,  but  logistics  tests  (low  observable  repair  and 
weapon  loading,  for  example)  were  not  completed  due  to 
delays  that  occurred  in  test  execution. 

•  Testing  of  timelines  to  meet  alert  launch  requirements 
showed  start-up  to  employment  capabilities  (both  air-to- 
air  and  air-to-ground)  exceeded  the  ORD  requirements 
(i.e.,  took  longer  than  required),  in  some  cases  up  to  several 
minutes.  Cold  alert  launches  performed  better  than  predicted 
(and  in  some  cases  met  requirements),  while  hot  launch 
times  were  worse  than  predicted.  The  program  has  no  plan 
to  address  these  requirements  during  SDD. 

•  The  program  did  not  fully  test  some  necessary  functions, 
such  as  landing  gear  operations.  Additionally,  some  major 
production  support  equipment  was  not  available  for  testing. 
Portable  enclosures  for  low-observable  restoration  did  not 
meet  expectations.  The  program  has  an  additional  test  period 
planned  for  February  2016  using  an  operational  aircraft. 

Cybersecurity  Operational  Testing 

•  In  accordance  with  DOT&E  and  DOD  policy,  the  JOTT 
developed  and  presented  a  cybersecurity  operational  test 
strategy  to  DOT&E  for  approval  in  February  2015.  This 
strategy  established  a  schedule  and  expectations  for 
cybersecurity  testing  of  the  JSF  air  system  through  the  end 
of  SDD  and  IOT&E  in  late  2017.  The  strategy  includes 
multiple  assessments  aligned  with  the  blocks  of  capability  as 
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the  program  delivers  them  to  the  field  in  both  the  air  vehicle 
and  AL1S.  The  test  teams  will  conduct  the  assessments 
on  fielded,  operational  equipment.  All  testing  requires 
coordination  from  the  JSF  Program  Executive  Officer,  via 
an  Interim  Authority  to  Test  (1ATT).  This  testing  is  OT&E; 
DOT&E  approves  the  plans  and  independently  reports 
results.  The  test  strategy  approved  by  DOT&E  includes 
end-to-end  testing  of  all  ALIS  components  and  the  F-35  air 
vehicle. 

•  The  Navy  conducted  a  Cooperative  Vulnerability  and 
Penetration  Assessment  (CVPA)  of  the  ALIS  Squadron 
Kit  2. 0.0. 2  aboard  the  USS  Wasp  from  May  26  through 
June  15,  2015.  Findings  were  mostly  administrative  in 
nature  and  the  test  team  recommended  changes  to  the 
procedures  for  updating  antivirus  signatures,  system  restoral, 
and  issuing  user  IDs  and  passwords  prior  to  systems 
becoming  operational  at  deployed  or  ship-based  locations. 

•  Starting  in  early  CY15,  the  JOTT  began  planning  CVPAs 
and  Adversarial  Assessments  (AA)  of  all  ALIS  components 
in  the  latest  configuration  to  be  fielded — ALIS  2. 0.1.1 — as 
well  as  the  F-35  air  vehicle  in  the  Block  2B  configuration. 
Consistent  with  the  strategy,  the  JOTT  planned  a  CVPA 
for  September  21  through  October  2,  2015,  and  an  AA  for 
November  9  -  20,  2015.  Only  the  ALIS  components  were  to 
be  tested  in  these  events,  with  an  air  vehicle  to  be  included 
in  a  future  test  event.  However,  the  test  teams  were  not 
able  to  complete  the  CVPA  as  planned  due  to  the  failure  of 
the  Program  Office  to  provide  an  1ATT.  According  to  the 
Program  Office,  an  1ATT  was  not  granted  due  to  insufficient 
understanding  of  risks  posed  to  the  operational  ALIS  systems 
by  cybersecurity  testing.  As  a  residt,  the  Program  Office 
directed  a  more  thorough  risk  assessment  and  restoration 
rehearsals  on  the  ALIS  systems  undergoing  testing  to 
improve  confidence  in  the  identified  risk  mitigations. 

•  To  recover  progress  on  the  test  strategy,  the  JOTT 
coordinated  with  cybersecurity  test  teams  for  the 
November  2015  AA  to  be  combined  with  a  CVPA.  However, 
the  program  approved  only  a  partial  1ATT,  which  allowed 

a  CVPA  of  the  ALIS  components  at  Edwards  AFB  and  a 
CVPA  of  the  Operational  Central  Point  of  Entry  (CPE) — a 
major  network  hub  in  the  overall  ALIS  architecture — to 
proceed.  Although  authorized,  the  AA  for  the  CPE  was 
not  accomplished  as  the  IATT  was  not  provided  in  time  for 
the  AA  team  to  make  arrangements  for  the  test.  Although 
significantly  limited  in  scope  relative  to  original  plans, 
the  testing  nonetheless  revealed  significant  cybersecurity 
deficiencies  that  must  be  corrected. 

•  An  end-to-end  enterprise  event,  which  links  each 
component  system,  including  the  air  vehicle,  is  required 
for  cybersecurity  operational  testing  to  be  adequate.  The 
test  teams  are  developing  the  needed  expertise  to  conduct  a 
technical  vulnerability  and  penetration  test  of  the  air  vehicle 
avionics  and  mission  systems.  Laboratory  simulators  at  the 
U.S.  Reprogramming  Lab  (USRL)  and  Lockheed  Martin 


might  be  suitable  environments  for  risk  reduction  and 
training,  but  will  not  take  the  place  of  testing  on  the  vehicle. 
The  Air  Force  Research  Laboratory  recently  published  an 
F-35  Blue  Book  of  potential  operational  vulnerabilities  that 
should  help  scope  future  air  vehicle  operational  testing. 

The  Program  Office  should  accelerate  the  actions  needed  to 
enable  cybersecurity  operational  testing  of  the  fielded  Block 
2B  and  Block  3i  systems  that  includes  both  ALIS  and  the  air 
vehicle. 

•  The  program  plans  to  develop  an  ALIS  test  laboratory, 
referred  to  as  the  Operationally  Representative  Environment, 
to  support  developmental  testing  and  risk  reduction  in 
preparation  for  future  operational  testing.  This  venue  should 
be  made  available  for  cybersecurity  testing  as  well,  but  will 
likely  not  be  an  adequate  venue  for  cybersecurity  testing  for 
IOT&E. 

Pilot  Escape  System 

•  In  201 1,  the  program  and  Services  elected  to  begin  training 
and  flight  operations  at  fielded  units  with  an  immature  pilot 
escape  system  by  accepting  risks  of  injury  to  pilots  during 
ejection.  These  risks  included  pilots  flying  training  missions 
with  ejection  seats  that  had  not  completed  full  qualification 
testing  and  flying  overwater  without  the  planned 
water-activated  parachute  release  system  (a  system  which 
automatically  releases  the  parachute  from  the  pilot’s  harness 
upon  entry  into  water).  Certain  risks  are  greater  for  lighter 
weight  pilots.  Recent  testing  of  the  escape  system  in  CY15 
showed  that  the  risk  of  serious  injury  or  death  are  greater  for 
lighter  weight  pilots  and  led  to  the  decision  by  the  Services 
to  restrict  pilots  weighing  less  than  136  pounds  from  flying 
the  F-35. 

•  Two  pilot  escape  system  sled  tests  occurred  in  July  and 
August  2015  that  resulted  in  failures  of  the  system  to 
successfully  eject  a  manikin  without  exceeding  neck 
loads/ stresses  on  the  manikin.  These  sled  tests  were  needed 
in  order  to  qualify  the  new  Gen  III  HMDS  for  flight  release. 

-  A  sled  test  in  July  on  a  103 -pound  manikin  with  a  Gen  111 
helmet  at  160  knots  speed  failed  for  neck  injury  criteria. 
The  program  did  not  consider  this  failure  to  be  solely 
caused  by  the  heavier  Gen  III  helmet,  primarily  due  to 
similarly  poor  test  results  having  been  observed  with 
Gen  II  helmet  on  a  103-pound  manikin  in  tests  in  2010. 

-  The  sled  test  was  repeated  in  August  20 1 5  using  a 
136-pound  manikin  with  the  Gen  III  helmet  at  160  knots. 
This  test  also  failed  for  neck  injury  criteria.  Similar 
sled  testing  with  Gen  II  helmets  in  2010  did  not  result  in 
exceedance  of  neck  loads  for  a  136-pound  pilot. 

•  After  the  latter  failure,  the  program  and  Services  decided  to 
restrict  pilots  weighing  less  than  136  pounds  from  flying  any 
F-35  variant,  regardless  of  helmet  type  (Gen  II  or  Gen  III). 
Pilots  weighing  between  136  and  165  pounds  are  considered 
at  less  risk  than  lighter  weight  pilots,  but  at  an  increased 
risk  (compared  to  heavier  pilots).  The  level  of  risk  was 
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labeled  “serious”  risk  by  the  Program  Office  based  on  the 
probability  of  death  being  23  percent  and  the  probability  of 
neck  extension  (which  will  result  in  some  level  of  injury) 
being  100  percent.  Currently,  the  program  and  the  Services 
have  decided  to  accept  the  risk  to  pilots  in  this  weight  range, 
although  the  basis  for  the  decision  to  accept  these  risks  is 
unknown. 

•  The  testing  showed  that  the  ejection  seat  rotates  backwards 
after  ejection.  This  results  in  the  pilot’s  neck  becoming 
extended,  as  the  head  moves  behind  the  shoulders  in 

a  “chin  up”  position.  When  the  parachute  inflates  and 
begins  to  extract  the  pilot  from  the  seat  (with  great  force), 
a  “whiplash”  action  occurs.  The  rotation  of  the  seat  and 
resulting  extension  of  the  neck  are  greater  for  lighter  weight 
pilots. 

•  The  Gen  III  helmet  weighs  5.1  pounds,  approximately 

6  ounces  more  than  the  Gen  II  helmet.  The  increased  weight 
is  primarily  due  to  the  larger/heavier  night  vision  camera 
optics.  The  program  has  a  weight  reduction  project  ongoing 
to  determine  if  approximately  5  ounces  can  be  eliminated 
in  the  Gen  III  helmet  by  reducing  structure  and  materials 
without  affecting  fit  or  optics. 

•  In  coordination  with  the  Program  Office,  the  ejection  seat 
contractor  funded  a  proof-of-concept  ejection  sled  test  in 
October  to  assess  the  utility  of  a  head  support  panel  (HSP),  a 
fabric  mesh  behind  the  pilots  head  and  between  the  parachute 
risers,  to  prevent  exceeding  neck  loads  during  the  ejection 
sequence  for  lighter  weight  pilots.  Based  on  the  initial 
results,  the  Program  Office  and  Services  are  considering  seat 
modifications  that  would  include  the  HSP,  but  they  may  take 
up  to  a  year  to  verify  improvement  and  install  them  onto 
aircraft. 

•  Additional  testing  and  analysis  are  also  needed  to  determine 
the  risk  of  pilots  being  harmed  by  the  transparency  removal 
system  (which  shatters  the  canopy  before,  and  in  order 

for,  the  seat  and  pilot  to  leave  the  aircraft)  during  ejections 
in  other  than  ideal,  stable  conditions  (such  as  after  battle 
damage  or  during  out-of-control  situations). 

•  The  program  began  delivering  F-35  aircraft  with  a 
water-activated  parachute  release  system  in  later  deliveries 
of  Lot  6  aircraft  in  2015.  This  system,  common  in  current 
fighter  aircraft,  automatically  jettisons  the  parachute  when 
the  pilot  enters  water  after  ejection  and  is  particularly 
beneficial  if  the  pilot  is  incapacitated  at  this  point. 

Progress  in  Modification  of  LRIP  Aircraft 

•  Modification  of  early  production  aircraft  is  a  major  endeavor 
for  the  program,  driven  by  the  large  degree  of  concurrency 
between  development  and  production.  Modifications  are 
dependent  on  the  production,  procurement,  and  installation 
of  modification  kits,  completed  either  at  the  aircraft  depot 
locations  or  at  the  field  units.  If  early  production  aircraft 
are  to  be  used  for  IOT&E,  as  has  been  planned,  the  program 
will  need  to  modify  them  in  order  to  provide  production 
representative  Block  3F  operational  test  aircraft  for  an 


adequate  IOT&E.  Current  projections  by  the  Program  Office 
show  that,  even  with  accelerated  contracting,  the  operational 
test  fleet  will  not  complete  modifications  until  April  2019. 
This  is  20  months  past  August  20 1 7,  the  date  currently 
planned  by  the  Program  Office  for  the  start  of  IOT&E. 

•  The  program  maintains  a  complex  modification  and 
retrofit  database  that  tracks  modifications  required  by  each 
aircraft,  production  break-in  of  modifications,  limitations 
to  the  aircraft  in  performance  envelope  and  service  life, 
requirements  for  additional  inspections  until  modifications 
are  completed,  and  operational  test  requirements  and 
concerns. 

-  Major  modifications  take  place  at  aircraft  depots  while 
depot  field  teams  will  travel  to  field  unit  to  complete  other 
modifications.  Additional  modifications  will  occur  while 
aircraft  undergo  unit-level  maintenance. 

-  Some  aircraft,  primarily  those  assigned  to  operational  test, 
will  undergo  modification  first  to  a  Block  2B  and  then  to  a 
Block  3F  configuration,  and  will  require  two  inductions  to 
an  aircraft  depot  for  several  months  each. 

•  Upgrading  F-35  aircraft  to  a  Block  2B  configuration 
includes  modifications  based  on  capability  and  life  limits  on 
hardware.  Major  modification  categories  include: 

-  Structural  life-limited  parts,  or  Group  1  modifications 

-  F-35B  Mode  4  operations,  including  a  modification  to  the 
Three  Bearing  Swivel  Module  (3BSM)  so  F-35B  aircraft 
can  conduct  unrestricted  Mode  4  operations 

-  On-Board  Inert  Gas  Generation  System  (OBIGGS),  which 
provides  the  upgraded  hardware  for  generating  adequate 
nitrogen-enriched  air  to  support  lightning  protection 
requirements  and  reduce  vulnerability  to  fuel  tank 
explosions  from  a  live  fire  event;  however,  the  aircraft  will 
need  additional  modifications  to  the  fuel  system  for  full 
lightning  and  vulnerability  protection 

-  Upgrades  to  ALIS  and  training  systems 

•  During  the  first  half  of  2015,  Marine  Corps  IOC  aircraft 
received  top  priority  for  Block  2B  modifications.  During  the 
second  half  of  2015,  the  program  prioritized  modification  of 
operational  test  aircraft. 

-  To  successfully  modify  Marine  Corps  aircraft  in  time  for 
IOC,  and  because  aircraft  modifications  frequently  took 
longer  than  projected,  the  program,  for  the  first  time,  sent 
Marine  Corps  aircraft  to  the  Air  Force  depot  at  Hill  AFB. 

-  Because  of  the  re-scoping  of  the  Block  2B  operational 
testing,  the  program  delayed  modifications  to  a  number 
of  aircraft  assigned  to  operational  test  squadrons.  As  of 
December  2015,  8  of  14  aircraft  assigned  to  operational 
test  squadrons  were  in  the  full  Block  2B  configuration, 
which  includes  the  OBIGGS  modification,  with  1  more 
undergoing  depot  modifications.  One  F-35B  is  not 
scheduled  to  complete  this  modification  until  June  2017. 
Twelve  of  the  14  aircraft  have  been  at  least  partially 
modified  to  the  Block  2B  configuration,  allowing  them  to 
fly  with  the  Block  2B  software. 
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Modifying  aircraft  to  a  Block  3F  configuration  includes 
completing  Block  2B  modifications.  Technical  Refresh  2 
(TR-2)  upgrades,  and  Block  3F  changes.  The  table  below 
shows  known  requirements  by  production  lot  of  aircraft  and 
the  number  of  those  that  are  authorized  and  scheduled  as  of 
July  2015.  Later  lots  of  aircraft  require  fewer  modifications 
because  of  changes  incorporated  into  the  production  line. 


Current  Program  Office  plans  for  modifications  show  that 
none  of  the  operational  test  aircraft  will  have  all  Block  3F 
modifications  completed  by  the  Program  Office’s  projected 
start  of  IOT&E  in  August  2017. 

-  The  program  awarded  an  undefinitized  contract  action 
(UCA)  for  new  TR-2  processors  in  September  2015  to 
support  Block  3F  retrofit  modifications  of  the  Block  2B 
operational  test  aircraft.  However,  the  TR-2  hardware 
packages  have  a  26-month  lead-time  which,  along  with 
other  required  changes  that  do  not  yet  have  approved 
engineering  or  hardware  solutions,  will  delay  the  complete 
modification  of  any  operational  test  aircraft  until  after 
IOT&E  is  scheduled  to  start. 

-  The  program  is  analyzing  options  to  reduce  this  timeline, 
including  seeking  authorization  outside  of  normal 
acquisition  practices  to  purchase  hardware  early,  taking 
components  from  the  production  line  before  installation 
occurs  for  use  on  operational  test  aircraft,  and  installing 
instrumentation  on  later  LRIP  aircraft  that  will  have 
already  received  this  hardware  during  production. 

-  The  majority  of  aircraft  assigned  to  operational  test 
squadrons  are  LRIP  3  and  4  aircraft,  which  require 
extensive  modifications  to  reach  a  Block  3F  configuration. 

The  program  has  had  difficulty  maintaining  the  planned 
induction  schedule  at  the  two  F-35  depots  located  at  MCAS 
Cherry  Point,  North  Carolina,  and  Hill  AFB,  Utah,  after 
structural  modifications  took  20  days  longer  than  planned  on 
early  inductees,  and  Lockheed  Martin  delivered  modification 
kits  late.  Transportation  issues  also  resulted  in  retrograde 
assets  not  shipping  in  a  timely  manner  for  repairs  and 
upgrades. 

-  At  MCAS  Cherry  Point,  early  F-35B  aircraft  inducted  took 
45  days  longer  than  projected  to  complete  modifications 
and,  as  of  July  2015,  the  depot  had  used  nearly  300  more 
cumulative  maintenance  days  than  projected  to  modify 
aircraft.  To  meet  Marine  Corps  IOC  requirements,  the 
program  sent  two  aircraft,  BF-3 1  and  BF-32,  to  Hill  AFB 
to  complete  structural  modifications.  Prior  to  this,  the 
program  had  not  scheduled  F-35  A  or  F-35B  aircraft  to 


complete  modifications  at  the  other  Service’s  depot.  As 
of  June  2015,  the  MCAS  Cherry  Point  depot  completed 
modifications  on  16  aircraft,  5  of  which  the  program 
needed  for  Marine  Corps  IOC. 

The  Hill  AFB  depot  has  stayed  closer  to  projections  on 
completing  modifications.  Although  early  inductees 
exceeded  the  planned  timeline,  later  aircraft,  including  the 
two  F-35B  aircraft,  have  completed 
modifications  in  less  time  than 
projected.  Fourteen  aircraft  have 
completed  modifications  at  this 
depot,  including  two  F-35B  aircraft 
needed  for  Marine  Corps  IOC.  Hill 
AFB,  which  began  the  year  with 
three  operational  docks,  expanded 
their  depot  capacity  to  eight  docks  in 
2015  by  accelerating  the  opening  of  four  of  these  docks  to 
reduce  the  risk  of  maintaining  the  modification  schedule. 

-  The  program  further  reduced  risk  to  the  modification 
schedule  by  employing  additional  field  teams  to  complete 
modifications  previously  planned  to  occur  during  aircraft 
inductions. 

-  By  July  2015,  both  depots  showed  improved  tracking  with 
the  depot  flow  plan. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  addressed 
two  of  the  previous  recommendations.  As  discussed  in  the 
appropriate  sections  of  this  report,  the  program  did  not,  and 
still  should: 

1.  Update  program  schedules  to  reflect  the  start  of  spin-up 
training  for  IOT&E  to  occur  no  earlier  than  the  operational 
test  readiness  review  planned  for  November  2017,  and  the 
associated  start  of  IOT&E  six  months  later,  in  May  2018. 

2.  Complete  lab  testing  of  the  mission  data  loads,  as  is  planned 
in  the  mission  data  optimization  operational  test  plan,  prior 
to  accomplishing  the  necessary  flight  testing  to  ensure  the 
loads  released  to  the  fleet  are  optimized  for  performance.  If 
mission  data  loads  are  released  to  operational  units  prior  to 
the  completion  of  the  lab  and  flight  testing  required  in  the 
operational  test  plan,  the  risk  to  operational  units  must  be 
clearly  documented.  Status:  Lab  testing  in  Block  2B  is  still 
in  work;  2B  build  fielded  to  operational  F-35B  units,  risk 
not  documented. 

3.  Complete  the  remaining  three  Block  2B  weapon  delivery 
accuracy  (WDA)  flight  test  events  in  a  way  that  ensures  full 
mission  systems  functionality  is  enabled  in  an  operationally 
realistic  manner. 

4.  Provide  adequate  resourcing  to  support  the  extensive 
validation  and  verification  requirements  for  the  Block  3 
VSim  in  time  for  IOT&E,  including  the  data  needed  from 
flight  test  or  other  test  venues. 

5.  Extend  the  full-up  system-level  (FUSL)  decontamination 
test  to  demonstrate  the  decontamination  system 
effectiveness  in  a  range  of  operationally  realistic 


KNOWN  BLOCK 3  IOT&E  MODIFICATION  REQUIREMENTS  IN  LOTS  3 THROUGH  9' 


Variant 

Lot  3 

Lot  4 

Lot  5 

Lot  6 

Lot  7 

Lot  8 

Lot  9 

F-35A 

1 24  (69) 

100(44) 

83  (32) 

38(15) 

15(2) 

10(1) 

2(1) 

F-35B 

1 30  (77) 

106  (56) 

82  (38) 

38(19) 

10(2) 

3(0) 

1  (0) 

F-35C 

- 

96  (43) 

80  (30) 

38(15) 

14(2) 

8(1) 

2(0) 

1 .  Numbers  in  parentheses  denote  authorized  and  scheduled  modifications. 


F-35JSF  81 


F  Y  1  5  DOD  PROGRAMS 


environments.  Status:  The  Program  Office  has  elected 
not  to  address  this  recommendation:  the  FUSL  test  will  be 
conducted  only  under  ambient  conditions  at  Edwards  AFB 
during  4QFY16  through  1QFY17  preventing  the  assessment 
of  this  system  in  other,  potentially  more  stressing  ambient 
conditions. 

6.  Ensure  adequate  testing  of  ALIS  software  upgrades  on 
operationally-representative  hardware  is  complete  prior  to 
fielding  to  operational  units. 

•  FY 15  Recommendations.  The  program  should: 

1 .  Acknowledge  schedule  pressures  that  make  the  start  of 
IOT&E  in  August  2017  unrealistic  and  adjust  the  program 
schedule  to  reflect  the  start  of  IOT&E  no  earlier  than 
August  2018. 

2.  The  Department  should  carefully  consider  whether 
committing  to  a  “block  buy,”  composed  of  three  lots 
of  aircraft,  is  prudent  given  the  state  of  maturity  of  the 
program,  as  well  as  whether  the  block  buy  is  consistent  with 
a  “fly  before  you  buy”  approach  to  defense  acquisition  and 
the  requirements  of  Title  10  United  States  Code. 

3.  Plan  and  program  for  additional  Block  3F  software  builds 
and  follow-on  testing  to  address  deficiencies  currently 
documented  from  Blocks  2B  and  3i,  deficiencies  discovered 
during  Block  3F  developmental  testing  and  during  IOT&E, 
prior  to  the  first  Block  4  software  release  planned  for  2020. 

4.  Significantly  reduce  post-mission  Ground  Data  Security 
Assembly  Receptacle  (GDR)  processing  times,  in  particular, 
decryption  processing  time. 

5.  Ensure  the  testing  of  Block  3F  weapons  prior  to  the  start 
of  IOT&E  leads  to  a  full  characterization  of  fire-control 
performance  using  the  fully  integrated  mission  systems 
capability  to  engage  and  kill  targets. 


6.  Complete  the  planned  climatic  lab  testing. 

7.  Provide  the  funding  and  accelerate  contract  actions  to 
procure  and  install  the  full  set  of  upgrades  recommended 
by  DOT&E  in  2012,  correct  stimulation  problems,  and  fix 
all  of  the  tools  so  the  U.S.  Reprogramming  Lab  (USRL) 
can  operate  efficiently  before  Block  3F  mission  data  load 
development  begins. 

8.  Complete  the  planned  testing  detailed  in  the 
DOT&E-approved  USRL  mission  data  optimization 
operational  test  plan  and  amendment. 

9.  Along  with  the  Navy  and  Marine  Corps,  conduct  an  actual 
operational  test  of  the  F-35B  onboard  an  L-class  ship 
before  conducting  a  combat  deployment  with  the  F-35B. 
This  test  should  have  the  full  Air  Combat  Element  (ACE) 
onboard,  include  ordnance  employment  and  the  full  use 
of  mission  systems,  and  should  be  equipped  with  the 
production-representative  support  equipment. 

10.  Develop  a  solution  to  address  the  modification  and  retrofit 
schedule  delays  for  production-representative  operational 
test  aircraft  for  IOT&E.  These  aircraft  must  be  similar  to,  if 
not  from  the  Lot  9  production  line. 

11.  Provide  developmental  flight  test  tracking  products  that 
clearly  show  progress  on  what  has  been  accomplished  and 
test  activity  remaining. 

12.  Develop  an  end-to-end  ALIS  test  venue  that  is  production 
representative  of  all  ALIS  components. 

13.  Ensure  the  necessary  authorizations  are  provided  in  time 
to  permit  operational  cybersecurity  testing  of  the  entire 
F-35  air  system,  including  the  air  vehicle,  as  planned  by  the 
operational  test  community. 

14.  Provide  dedicated  time  on  representative  air  vehicles  to 
complete  Joint  Technical  Data  (JTD)  verification. 
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Global  Command  and  Control  System  -  Joint  (GCCS-J) 


Executive  Summary 

•  In  FY15,  Defense  Information  Systems  Agency  (DISA) 
development  of  Global  Command  and  Control  System  -  Joint 
(GCCS-J)  focused  on  fixing  cybersecurity  vulnerabilities, 
implementing  high-priority  capability  enhancements,  and 
software  defect  corrections  to  both  the  GCCS-J  Global 
(referred  to  as  Global)  and  Joint  Operation  Planning  and 
Execution  System  (JOPES). 

Global 

•  J1TC  conducted  a  Global  v5.0  pilot  test  in  January  2015 
in  preparation  for  entering  operational  testing.  Global 
v5.0  failed  to  satisfy  operational  test  entrance  criteria,  and 
DISA,  with  concurrence  of  the  operational  community, 
subsequently  cancelled  Global  v5.0  in  order  to  reduce 
risk  to  Global  v4.3  Update  1  sustainment  and  Global  v6.0 
development. 

•  DISA  developed  Global  v4.3  Update  1  Emergency 
Release  1  to  resolve  an  operational  deficiency  discovered 
in  the  fielded  Global  v4.3  Update  1  software.  This  release 
also  included  some  of  the  improvements  originally  planned 
for  the  cancelled  Global  v5.0.  The  Joint  Interoperability 
Test  Command  (JITC)  and  DISA  tested  Global  v4.3  Update 
1  Emergency  Release  1  in  April  2015. 

-  GCCS-J  Global  v4.3  Update  1,  with  Emergency 
Release  1,  remains  effective  for  use  in  higher  echelons. 
Testing  of  Global  v4.3  Update  1  for  use  in  lower 
echelons  will  occur  in  FY16  as  part  of  Air  Operations 
Center  -  Weapons  System  operational  testing. 

-  GCCS-J  v4.3  is  operationally  suitable.  System 
installation,  help  desk,  training,  and  availability  are  all 
acceptable. 

-  GCCS-J  v4.3  Update  1  is  not  survivable.  DISA  has 
not  fixed  all  vulnerabilities  identified  by  the  National 
Security  Agency  (NSA)  cybersecurity  testing,  and 
additional  vulnerabilities  have  been  identified  by 
cybersecurity  testing  as  part  of  major  Combatant 
Command  exercises. 

JOPES 

•  DISA  developed  JOPES  v4.2.0.3  Emergency  Release  4 
to  implement  Global  Force  Management  capabilities 
and  to  implement  Operational  Plan  Relevancy  codes. 

JITC  conducted  an  operational  test  of  JOPES  v4.2.0.3 
Emergency  Release  4  in  June  2015. 

-  JOPES  v4.2.0.3  Emergency  Release  4  is  operationally 
effective  and  suitable.  Users  successfully  employed  new 
Global  Force  Management  capabilities  and  completed  all 
mission  tasks. 

-  The  cyber  survivability  of  JOPES  v4.2.0.3  Emergency 
Release  4  has  not  yet  been  tested. 


Global  Infrastructure 


CommonOperational 
Picture 


JointOperation  Planning  & 


Cross  Domain  Services 
Modernization  Initiative 

GCCS  CDS  Ovorvltw 


System 

•  GCCS-J  consists  of  hardware,  software  (both  commercial 
off-the-shelf  and  government  off-the-shelf),  procedures, 
standards,  and  interfaces  that  provide  an  integrated  near 
real-time  picture  of  the  battlespace  that  is  necessary  to  conduct 
joint  and  multi-national  operations. 

•  GCCS-J  consists  of  a  client/server  architecture  using 
open-systems  standards,  government-developed  military 
planning  software,  and,  increasingly,  use  of  World  Wide  Web 
technology. 

•  GCCS-J  consists  of  two  components,  GCCS-J  Global  and 
JOPES. 

Global  (Force  Protection,  Situational  Awareness,  and 
Intelligence  applications) 

•  Global  v4.3  Update  1,  Emergency  Release  1  is  the  currently 
fielded  version.  DISA  developed  Global  v4.3  Update  1 

to  implement  high-priority  intelligence  mission  updates 
to  the  Theater  Ballistic  Missile  correlation  systems,  Joint 
Targeting  Toolbox,  and  Modernized  Integrated  Database. 
The  update  also  resolved  49  defects  affecting  other  parts 
of  the  system  and  implemented  security  lockdown  scripts 
and  Information  Assurance  Vulnerability  Alert  updates. 
Emergency  Release  1  resolved  an  operational  deficiency 
discovered  in  the  fielded  Global  v4.3  Update  1  software. 

•  Global  v5.0.  DISA  developed  Global  v5.0  to  introduce 
updates  and  new  features  to  the  Cross-Functional/ 
Infrastructure,  Situational  Awareness,  and  Integrated 
Imagery  and  Intelligence  mission  areas.  DISA  also  fixed 
33  problems,  all  of  which  had  approved  operational 
workarounds.  Poor  test  results  in  2015  induced  DISA  to 
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cancel  Global  v5.0  and  instead  focus  on  development  of 
Global  v6.0. 

•  Global  v6.0.  This  version  will  contain  features  from  v5.0 
and  implement  an  Agile  Client  as  the  primary  GCCS-J 
user  interface  to  allow  removal  of  the  global  client  from 
the  system  baseline.  DISA  is  also  modernizing  Global 
interfaces  to  provide  greater  flexibility  for  information 
sharing  with  external  systems. 

JOPES  (Force  Employment,  Projection,  Planning,  and 

Deployment/Redeployment  applications) 

•  JOPES  v4.2.0.3  Emergency  Release  4  is  the  currently 
fielded  version.  DISA  developed  JOPES  v4.2.0.3 
Emergency  Release  4  to  implement  Global  Force 
Management  capabilities  and  to  implement  Operational 
Plan  Relevancy  codes.  Force  Tracking  Number  and 
Deployment  Order  information  were  added  to  the  system, 
as  well  as  an  ability  to  identify  and  query  operationally 
relevant  plans.  DISA  also  corrected  seven  critical 
deficiencies. 

Mission 

•  Joint  Commanders  utilize  the  GCCS-J  to  accomplish 

command  and  control. 

Global 

•  Commanders  use  Global  to: 

-  Link  the  National  Command  Authority  to  the  Joint  Task 
Force,  Component  Commanders,  and  Service  unique 
systems  at  lower  levels  of  command 

-  Process,  correlate,  and  display  geographic  track 
information  integrated  with  available  intelligence  and 


environmental  information  to  provide  the  user  a  fused 
battlespace  picture 

-  Provide  Integrated  Imagery  and  Intelligence  capabilities 
(e.g.  battlespace  views  and  other  relevant  intelligence) 
into  the  common  operational  picture  and  allow 
commanders  to  manage  and  produce  target  data  using 
the  Joint  Tactical  Terminal 

-  Provide  a  missile  warning  and  tracking  capability 

•  Air  Operations  Centers  use  Global  to: 

-  Build  the  air  picture  portion  of  the  common  operational 
picture  and  maintain  its  accuracy 

-  Correlate  or  merge  raw  track  data  from  multiple  sources 

-  Associate  raw  Electronics  Intelligence  data  with  track 
data 

-  Perform  targeting  operations 

JOPES 

•  Commanders  use  JOPES  to: 

-  Translate  policy  decisions  into  operations  plans  that 
meet  U.S.  requirements  to  employ  military  forces 

-  Support  force  deployment,  redeployment,  retrograde, 
and  re -posturing 

-  Conduct  contingency  and  crisis  action  planning 

Major  Contractors 

•  Government  Integrator:  Defense  Information  Systems  Agency 

(DISA) 

•  Software  Developers: 

-  Northrop  Grumman  -  Arlington,  Virginia 

-  Leidos  -  Arlington,  Virginia 

-  Pragmatics  -  Arlington,  Virginia 


Activity 

Global 

•  In  September  2014,  DISA  approved  Global  v4.3  Update  1 
fielding,  based  on  results  from  developmental  and 
operational  testing  conducted  in  2014.  DISA  fixed  eight  of 
nine  cybersecurity  vulnerabilities  identified  as  part  of  NSA 
cybersecurity  testing  shortly  after  fielding. 

•  JITC  conducted  the  Global  v5.0  pilot  test  at  U.S.  Special 
Operations  Command,  MacDill  AFB,  Florida,  from 
January  7-9,  2015,  to  assess  the  systems  readiness  to  enter 
operational  test.  Global  v5.0  failed  to  satisfy  operational 
test  entrance  criteria,  and  DISA,  with  concurrence  of  the 
operational  community,  subsequently  cancelled  Global  v5.0 
in  order  to  reduce  risk  to  Global  v4.3  Update  1  sustainment 
and  Global  v6.0  development. 

•  DISA  developed  Global  v4.3  Update  1  Emergency 
Release  1  to  resolve  an  operational  deficiency  discovered 
in  the  fielded  Global  v4.3  Update  1  software.  This  release 
also  included  some  of  the  improvements  originally  planned 
for  Global  v5.0.  In  April  2015,  JITC  and  DISA  conducted  a 
level  1  operational  test  of  Global  v4.3  Update  1  Emergency 
Release  1  in  accordance  with  a  DOT&E-approved  policy 
that  did  not  require  a  DOT&E-approved  test  plan. 
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•  On  January  7,  2015,  the  Joint  Staff  released  a  memorandum 
directing  a  comprehensive  review  of  Global  critical  and 
non-critical  interface  requirements.  The  Joint  Staff  directed 
the  review  to  confirm  that  Service  member  data  exchange 
requirements  in  support  of  operational  missions  were 
being  met.  This  review  helped  update  the  correct  critical 
interfaces,  and  the  results  of  these  updates  will  drive  the 
content,  development,  and  testing  of  Global  v6.0. 

JOPES 

•  In  June  2015,  JITC  conducted  a  level  1  operational  test 
of  JOPES  v4.2.0.3  Emergency  Release  4  in  accordance 
with  a  DOT&E-approved  policy  that  did  not  require 

a  DOT&E-approved  test  plan.  This  testing  included 
interface  testing  with  Defense  Readiness  Reporting 
System  -  Strategic. 

Assessment 

Global 

•  GCCS-J  v4.3  Update  1,  with  Emergency  Release  1,  is 
effective  for  use  in  higher  echelons. 
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-  Further  operational  testing  is  required  to  determine  the 
effectiveness  for  use  in  lower  echelons,  such  as  Air 
Operations  Centers.  The  interface  requirement  updates 
directed  by  the  Joint  Staff  will  assist  the  Air  Operations 
Center  test  community  in  assessing  effectiveness  at 
lower  echelons. 

-  Developmental  testing  of  Global  v4.3  Update  1  is 
planned  for  the  Air  Operations  Center  community  in 
October  2015,  with  operational  testing  planned  for 
January  2016. 

•  GCCS-J  Global  v4.3  Update  1  is  not  survivable. 

-  DISA  has  fixed  eight  of  nine  significant  vulnerabilities 
identified  by  NS  A  cybersecurity  testing;  however, 
one  significant  vulnerability  remains.  Additional 
GCCS-J  vulnerabilities  have  been  identified  by 
DOT&E-sponsored  cybersecurity  testing  during  major 
Combatant  Command  exercises. 

•  GCCS-J  v4.3  Update  1  is  operationally  suitable.  System 
installation,  help  desk,  training,  and  availability  are  all 
acceptable. 

JOPES 

•  JOPES  v4.2.0.3  Emergency  Release  4  is  operationally 
effective. 

-  Users  successfully  employed  new  Global  Force 
Management  capabilities  and  completed  all  mission 
tasks. 

-  All  Combatant  Commands  experienced  data  exchange 
failures  linked  to  either  initial  subscription  or  interface 
maintenance.  The  Joint  Staff,  in  coordination 

with  the  Combatant  Commands,  updated  existing 
standard  operating  procedures  and  identified  roles  and 
responsibilities  to  manage  the  processes. 

-  The  JOPES  program  manager  resolved  all  high-priority 
problem  reports,  and  JITC  did  not  discover  any  new 
problems  during  operational  testing. 


-  JOPES  demonstrated  effective  end-to-end  data 
exchanges  with  the  new  Joint  Capabilities  Requirements 
Manager,  as  well  as  with  Global  Combat  Support 
System  -  Joint  and  the  Deliberate  and  Crisis  Action 
Planning  and  Execution  Segments. 

•  JOPES  v4.2.0.3  Emergency  Release  4  is  operationally 
suitable. 

-  System  administrators  successfully  installed  and 
configured  the  system.  The  Combatant  Commands 
validated  the  updated  standard  operating  procedures 
to  support  the  new  Joint  Capabilities  Requirements 
Manager  to  JOPES  interface. 

-  The  system  was  available  throughout  the  test  period,  and 
users  did  not  notice  any  degradation  of  performance  or 
usability. 

•  The  cybersecurity  of  JOPES  v4.2.0.3  Emergency  Release  4 
has  not  been  tested. 

Recommendations 

•  Status  of  Previous  Recommendations.  DISA  has  addressed 
one  of  the  two  previous  FY14  recommendations.  However, 
DISA  still  needs  to  conduct  cybersecurity  testing  of  GCCS 
Global  v4.3  Update  1  in  an  operational  environment  to  assess 
protect,  detect,  react,  and  recover  capabilities. 

•  FY 15  Recommendations.  DISA  should: 

1.  Correct  any  remaining  major  cybersecurity  vulnerabilities 
identified  either  by  the  NSA  assessment  of  the  GCCS-J 
v4.3  baseline,  or  during  subsequent  Combatant  Command 
exercises. 

2.  Conduct  cybersecurity  testing  of  both  GCCS  Global 
v4.3  Update  1  Emergency  Release  1  and  the  JOPES 
v4.2.0.3  Emergency  Release  4  baselines  in  operational 
environments,  fix  any  cybersecurity  vulnerabilities 
identified,  and  conduct  appropriate  regression  testing. 


GCCS-J 
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Joint  Information  Environment  (JIE) 


B/P/C/S 


B/P/C/S 


GSU 


B/P/C/S  -  Base,  Post,  Camp,  and  Station 

BAN  -  Base  Area  Network 

CAP  -  Cloud  Access  Point 

CDC  -  Core  Data  Center 

CDS  -  Cross  Domain  Solution 

CSAAC  -  Cyber  Situational  Awareness  Analytic  Cloud 
Data  Svcs  -  Data  Services 
DMZ  -  Demilitarized  Zone 

Ent  Svcs  -  DOD  Enterprise  Services  (‘Extended) 

EOC  -  Enterprise  Operations  Center 


GEOC  -  Global  Enterprise  Operations  Center 

GSU  -  Geographic  Separated  Unit 

IAP  -  Internet  Access  Point 

IdAM  -  Identitiy  and  Access  Managment 

IG  -  Installation  Gateway 

IPN  -  Installation  Processing  Nodes 

ISN  -  Installation  Service  Node 

JMN  -  JIE  Management  Network 

JMS  -  JIE  Management  System 

JRSS  -  Joint  Regional  Security  Stacks 


MPGW  -  Mission  Partner  Gateway 

MPLS  -  Multiprotocol  Label  Switching 

NOSC  -  Network  Operations  &  Security  Center 

SAT  -  Satellite  Communications  Gateway 

SPPN  -  Special  Purpose  Processing  Node 

SSA  -  Single  Security  Architecture 

Svc  Apps  -  DOD  Component  Applications 

TPN  -  Tactical  Processing  Node 

UC  -  Unified  Capabilities 

Web  Apps  -  Web  Applications 


Executive  Summary 

•  The  Joint  Information  Environment  (JIE)  is  not  a  program  of 
record,  and  to  date,  the  Defense  Information  Systems  Agency 
(DISA),  Joint  Interoperability  Test  Command  (J1TC),  and 
Services  have  not  conducted  any  operational  testing  of  the  JIE 
infrastructure  or  components. 

•  The  JIE  effort  lacks  an  overarching  systems  integration 
process  or  program  executive  organization  to  manage  cost, 
drive  schedule,  and  monitor  performance  factors.  The 
European  JIE  early  operational  assessment,  originally 
scheduled  for  March/ April  2014,  continues  to  slip  due  to 
Joint  Regional  Security  Stack  (JRSS)  integration  complexity, 
lack  of  overall  schedule  discipline,  and  Service-influenced 
funding  priorities;  the  DOD  Chief  Information  Officer  (CIO) 
has  shifted  its  near-term  focus  for  JRSS  to  the  Southern 
Continental  United  States  (CONUS). 

•  U.S.  Cyber  Command  established  the  Joint  Force 
Headquarters  DOD  Information  Networks  (JDOC)  and  the 
JIE  Executive  Committee  (EXCOM)  designated  the  DISA 
Operation  Centers  in  Europe  and  the  Pacific  as  Enterprise 
Operations  Centers  (EOCs)  to  support  JIE  network 
management,  data  centers,  Internet  gateways,  JRSS,  and 


cybersecurity.  The  Joint  Operations  Steering  Group  (JOSG) 
developed  EOC  continuity  of  operations  plans  to  better 
manage  networks,  and  coordinate  and  communicate  with 
Service  operations  centers.  DISA  established  EOCs  in  the 
European  and  Pacific  theaters  that  will  conduct  mission 
support  and  information  technology  operations. 

•  DOT&E  is  working  with  DISA  and  J1TC  to  plan  for  an  early 
operational  assessment  of  JIE  in  late  FY16. 

•  U.S.  Cyber  Command  continues  to  refine  the  selection  criteria 
for  the  CONUS  EOCs  as  well  as  the  JDOC/EOC  global  and 
regional  situational  awareness  requirements.  In  October 
2015,  the  JIE  EXCOM  endorsed  U.S.  Cyber  Command’s 
proposal  that  DISA  CONUS  be  the  interim  EOC  to  support  the 
implementation  of  JRSS. 

Capability  and  Attributes 

•  The  DOD  CIO  has  prioritized  areas  of  modernization  for  the 
DOD  components  to  implement  as  the  foundational  steps 

to  realize  the  JIE.  The  DOD  CIO’s  areas  of  modernization 
include  the  following: 
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-  Network  Modernization  of  optical  carrier  upgrades  and 
Multi-Protocol  Label  Switching  (MPLS)  routers 

-  The  JRSS,  the  Joint  Management  System  for  the  JRSS  and 
Cyber  Situational  Awareness  Capabilities 

-  Computing  Environment,  which  includes  Commercial 
Cloud,  Cloud  Access  Points,  and  milCloud 

-  The  Mission  Partner  Environment  for  coalition/partner 
information  sharing  and  the  Mission  Partner  Gateways 

-  Mobility  for  the  unclassified  and  classified  capabilities 

•  The  JIE  is  envisioned  as  a  shared  information  technology 
construct  for  DOD  to  reduce  costs,  improve  and  standardize 
physical  infrastructure,  increase  the  use  of  enterprise  services, 
improve  information  technology  effectiveness,  and  centralize 
the  management  of  network  security.  The  Joint  Staff  specifies 
the  following  enabling  characteristics  for  the  JIE  capability: 

-  Transition  to  centralized  data  storage 

-  Rapid  delivery  of  integrated  enterprise  services  (such  as 
email  and  collaboration) 

-  Real-time  cybersecurity  awareness 

-  Scalability  and  flexibility  to  provide  new  services 

-  Use  of  common  standards  and  operational  techniques 

-  Transition  to  a  single  security  architecture 

•  The  DOD  plans  to  achieve  these  goals  via  the  following 
interrelated  initiatives: 

-  Consolidation  of  applications  and  data  into  the  cloud  or 
centralized  data  centers  at  the  regional  or  global  level, 
which  are  not  segregated  by  military  Service. 


-  Establishment  of  enterprise  operation  centers  to  centralize 
network  management  and  defense. 

-  Upgrade  of  the  physical  infrastructure  to  include  MPLS, 
which  enables  higher  bandwidth/throughput,  better 
security,  and  faster  routing  capabilities. 

-  Implementation  of  JRSS  hardware  and  other  security 
constructs  as  part  of  a  single  security  architecture.  These 
will  establish  a  federated  network  structure  with 
standardized  identity  and  access  management,  as  well  as 
centralized  defensive  cyber  operations. 

•  JIE  is  not  a  program  of  record,  but  is  being  governed  by 
the  DOD  CIO,  with  DISA  as  the  principal  integrator  for 
services  and  testing.  An  EXCOM,  chaired  by  the  DOD  CIO, 
U.S.  Cyber  Command,  and  the  Joint  Staff  J6,  provides  JIE 
direction,  goals  and  objectives,  oversight,  and  accountability. 

•  The  initial  implementation  of  the  JIE  is  underway  in 
the  United  States  and  in  the  European  theater  with  the 
establishment  of  the  first  JRSS  capabilities,  JDOC  and  EOCs, 
and  the  European  data  centers.  Installations  are  ongoing  in 
Europe  with  tentative  implementation  and  cutover  dates  in 
June/July  2016.  Additional  preparation  efforts  are  ongoing  in 
the  Pacific,  Southwest  Asia,  and  CONUS. 


Activity 

•  The  JIE  EXCOM  rescheduled  an  early  operational  assessment 
of  the  European  theater  capabilities  originally  planned 

for  March  2014  to  4QFY16  or  later  to  accommodate  the 
engineering,  installation,  and  implementation  of  the  initial 
JRSS  and  MPLS  capabilities.  The  DOD  CIO  has  shifted  its 
near-term  focus  for  JRSS  to  the  Southern  CONUS. 

•  JITC  developed  an  evaluation  framework  that  maps  testable 
JIE  metrics  back  to  the  requirements  and  high  level  objectives. 

•  U.S.  Cyber  Command  established  the  JDOC  and  the  JIE 
EXCOM  designated  the  DISA  Operation  Centers  in  Europe 
and  the  Pacific  as  EOCs  to  support  JIE  network  management, 
data  centers,  Internet  gateways,  JRSS,  and  cybersecurity. 

•  The  JOSG  developed  EOC  continuity  of  operations  plans  to 
better  manage  networks,  and  coordinate  and  communicate  with 
Service  operations  centers. 

•  DISA  established  EOCs  in  the  European  and  Pacific  theaters 
that  will  conduct  mission  support  and  information  technology 
operations. 

•  In  May  and  July  2015,  the  Army,  Air  Force,  DISA,  and  JITC 
conducted  JRSS  lab-based  tests  at  Fort  Meade,  Maryland,  and 
Joint  Base  San  Antonio,  Texas. 

•  Developmental  and  laboratory  testing  continues  at  initial  JRSS 
sites  at  Fort  Hood  and  Joint  Base  San  Antonio,  Texas,  and 

the  DISA  Enterprise  Services  Lab  at  Fort  Meade,  Maryland. 


Current  testing  focuses  on  system  functionality;  however,  a 
Cyber  Protection  Team  (CPT)  conducted  a  hunt  mission  to 
find  outstanding  vulnerabilities  in  the  operational  management 
network  and  the  Joint  Base  San  Antonio  JRSS  from  March  to 
May  2015. 

•  In  October  2015,  the  JIE  EXCOM  approved  the  JIE  high-level 
objectives  and  initial  performance  metrics. 

•  U.S.  Cyber  Command  continues  to  refine  the  selection  criteria 
for  the  CONUS  EOCs  as  well  as  the  JDOC/EOC  global  and 
regional  situational  awareness  requirements.  In  October  2015, 
the  JIE  EXCOM  endorsed  U.S.  Cyber  Command’s  proposal 
that  DISA  CONUS  be  the  interim  EOC  to  support  the 
implementation  of  JRSS. 

•  DOT&E,  USD(AT&L),  DOD  CIO,  the  Services,  DISA, 
and  JITC  repurposed  the  JIE  T&E  working-level  Integrated 
Product  Team  into  an  overarching  working  group  to  better 
synchronize  the  test  preparations  and  ongoing  activities. 

•  A  CPT  and  the  Army’s  Advanced  Research  Lab  conducted 
a  comprehensive  vulnerability  and  penetration  assessment 
against  the  JRSS  in  the  Fort  Meade  labs  in  4QFY15. 

•  JITC  conducted  the  JRSS  version  1.0  Initial  Operational 
Assessment  with  a  Red  Team  to  assess  Army  and  Air  Force 
operations  in  December  2015. 
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Assessment 

•  The  JIE  is  not  a  program  of  record,  and  to  date,  DISA,  J1TC, 
and  Services  have  not  conducted  any  operational  testing  of  the 
JIE  infrastructure  or  components. 

•  The  DOD  CIO  is  the  lead  for  JIE  governance;  however,  the 
JIE  effort  lacks  an  overarching  systems  integration  process 
or  program  executive  organization  to  manage  cost,  drive 
schedule,  and  monitor  performance  factors.  The  European 
JIE  early  operational  assessment  continues  to  slip  due  to  JRSS 
integration  complexity,  lack  of  overall  schedule  discipline,  and 
Service-influenced  funding  priorities.  While  the  near-term 
focus  for  JRSS  has  shifted  to  the  Southern  CONUS,  advanced 
planning  for  future  capability  deployment  and  operational  tests 
has  not  fully  matured. 

•  No  operational  test  data  are  available  at  this  point. 

•  Current  testing  focuses  on  system  functionality;  DISA  has  yet 
to  schedule  full  cybersecurity  testing  or  operational  testing  of 
the  JRSS. 

•  DOT&E  is  working  with  DISA  and  J1TC  to  plan  for  an  early 
operational  assessment  of  JIE  in  late  FY16. 

•  DISA  and  J1TC  scheduled  a  lab-based  Computer  Network 
Defense  Exercise  for  mid-October  2015  but  it  was  delayed  due 
to  other  assessment  activities.  The  National  Security  Agency 
plans  to  conduct  a  cybersecurity  deep-dive  assessment  in 
FY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  DOD  CIO  and 
Director  of  DISA  continue  to  address  the  previous  FY14 


recommendations  to  develop  adequate  test  schedules  and  plans 
for  anticipated  future  test  events  in  FY16. 

FY15  Recommendations.  The  DOD  CIO,  JIE  EXCOM,  and 
DISA  should: 

1.  Establish  an  overarching  JIE  program  executive  to 
integrate  the  system  efforts  and  oversee  cost,  schedule,  and 
performance. 

2.  Consider  managing  the  key  JIE  capabilities/components 
with  program  managers. 

3.  Continue  to  develop  an  overarching  test  strategy  that 
encompasses  not  only  the  upcoming  testing  of  JIE,  but 
also  defines  the  key  issues  and  concepts  to  be  tested  in 
subsequent  tests  and  assessments.  Such  a  plan  should 
address  the  following  areas  of  interest: 

■  Overarching  T&E  framework  and  critical  test  issues 

■  The  role  of  both  lab  and  fielded  equipment  tests  in 
resolving  those  critical  issues 

■  Estimated  schedules  for  test  events  and  key  issues  to  be 
tested 

■  Evaluation  criteria  and  any  relevant  implementation 
decision  points 

■  Resources  required 

■  The  role  of  the  Services  and  Service-sponsored 
Operational  Test  Agencies 
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Joint  Warning  and  Reporting  Network  (JWARN) 


Executive  Summary 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  FOT&E  of  the  Joint  Warning  and 
Reporting  Network  (JWARN)  Increment  1  hosted  on  the 
Global  Command  and  Control  System  (GCCS)  -  Maritime 
(GCCS-M)  Force  Level  from  April  to  June  2015  onboard 
the  USS  Ronald  Reagan  (CVN  76),  USS  John  C.  Stennis 
(CVN  74),  and  JWARN  hosted  on  GCCS  -  Joint  at  the  Navy 
Third  Fleet  Maritime  Operations  Center  (MOC). 

•  JWARN  is  an  operationally  effective  tool  for  the  Navy  to 
provide  timely  hazard  warning  to  Navy  ships  and  other 
Service  units  operating  18  kilometers  or  more  away  from  the 
initial  chemical,  biological,  radiological,  and  nuclear  (CBRN) 
release  to  institute  force  protective  actions  before  encountering 
the  hazard.  Units  that  are  less  than  18  kilometers  from  the 
release  should  be  warned  by  other  means,  such  as  chat  or 
radio. 

•  Tactical  Action  Officers  were  able  to  use  JWARN  information 
to  make  operational  decisions  and  recommendations  to  ship 
commanders  and  share  CBRN  hazard  plots  within  the  Combat 
Direction  Center  and  with  other  units  via  the  GCCS-M 
Common  Operating  Picture  Synchronization  Tool. 

•  Testers  were  unable  to  exploit  JWARN  on  the  Consolidated 
Afloat  Network  Enterprise  Services  (CANES)  network  or 
GCCS-M  client  during  remote  cyber-attack  testing.  JWARN 
demonstrated  a  97  percent  operational  availability  and  met  the 
requirement  for  100  hours  Mean  Time  Between  Operational 
Mission  Failure. 

•  Current  plans  for  sustaining  JWARN  interoperability  in  a 
complex  network  operating  environment  are  not  adequate  to 
sustain  the  JWARN  capability  over  time. 

System 

•  JWARN  is  a  joint  automated  CBRN  warning,  reporting, 
and  analysis  software  tool  that  resides  on  joint  and  Service 
command  and  control  systems  including  the  GCCS  -  Army, 
GCCS  -  Joint,  GCCS  -  Maritime,  and  Command  and  Control 
Personal  Computer/Joint  Tactical  Common  Workstation. 


JWARN  screen  shot  depicting  CBRN  hazard  overlay  on  the  common  operational  map 

•  JWARN  software  automates  the  NATO  CBRN  warning 
and  reporting  process  to  increase  the  speed  and  accuracy  of 
information. 

•  JWARN  uses  the  common  operating  picture  of  the  host 
command  and  control  system  or  computing  environment  to 
display  the  location  of  CBRN  events  and  the  predicted  or 
actual  location  of  hazards. 

•  JWARN  is  an  application  on  the  GCCS-M,  and  is 
interoperable  with  the  ship’s  tactical  network,  the  CANES. 

Mission 

JWARN  operators  support  the  commander’s  force  protection  and 

operational  decisions  by: 

•  Providing  analysis  of  potential  or  actual  CBRN  hazard  areas 
based  on  operational  scenarios  or  sensor  and  observer  reports 

•  Identifying  affected  units  and  operating  areas 

•  Transmitting  warning  reports 

Major  Contractor 

Northrop  Grumman  Mission  Systems  -  Orlando,  Florida 


Activity 

•  COTF  conducted  FOT&E  of  the  JWARN  Increment  1  hosted 
on  the  GCCS-M  Force  Level  from  April  to  June  2015,  aboard 
the  USS  Ronald  Reagan  (CVN  76),  USS  John  C.  Stennis 
(CVN  74),  and  JWARN  hosted  on  GCCS  -  Joint  at  the  Navy 
Third  Fleet  MOC.  Testing  was  conducted  in  accordance  with 
the  DOT&E-approved  test  plan. 

•  FOT&E  consisted  of: 

-  Adversarial  intelligence  collection 


-  Local  and  remote  reconnaissance  and  cyber  attacks 

-  Initial  operator  training 

-  JWARN  installation  and  configuration  at  each  test  location 

-  Computer-based  refresher  training  immediately  preceding 
the  operational  test 

-  A  logistics  and  maintenance  demonstration 

•  A  three-day  operational  test  in  which  JWARN  operators 
received  reports  of  CBRN  incidents  and  used  JWARN  to 
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analyze  the  information  and  generate  hazard  plots  and  warning 
messages  sent  to  units  at  risk  of  exposure. 

•  COTF  collected  supplemental  test  data  on  the  time  it  takes 
to  achieve  various  levels  of  mission  oriented  protective 
posture  in  response  to  a  CBRN  threat  during  an  operational 
exercise  aboard  the  USS  Theodore  Roosevelt  (CVN  71)  in 
September  2014  and  February  2015. 

Assessment 

•  JWARN  is  an  operationally  effective  tool  for  the  Navy  to 
provide  timely  hazard  warning  to  Navy  ships  and  other  Service 
units  operating  18  kilometers  or  more  away  from  the  initial 
CBRN  release  to  institute  force  protective  actions  before 
encountering  the  hazard.  Units  that  are  less  than  18  kilometers 
from  the  release  should  be  warned  by  other  means,  such  as 
chat  or  radio. 

•  Tactical  Action  Officers  were  able  to  use  JWARN  information 
to  make  operational  decisions  and  recommendations  to  ship 
commanders  and  share  CBRN  hazard  plots  within  the  Combat 
Direction  Center  and  with  other  units  via  the  GCCS-M 
Common  Operating  Picture  Synchronization  Tool. 

•  Twenty-one  percent  of  hazard  warnings  (10  of  47)  were  not 
received  in  time  to  support  force  protection  due  to  CANES 
network  problems  or  long  message  transmission  times. 

•  JWARN  demonstrated  a  97  percent  operational  availability. 
Down  time  resulted  from  the  need  to  reboot  the  client 
computer  and  network  failures.  There  were  no  JWARN 
software  failures  during  118  hours  of  operation. 

•  The  local  and  remote  cyber  reconnaissance  did  not  expose 
significant  vulnerabilities  in  the  ship’s  network  or  the 
GCCS-M  client,  which  hosts  JWARN.  Testers  were  unable 
to  exploit  JWARN  on  the  CANES  network  or  GCCS-M  client 
during  remote  cyber-attack  testing. 


•  Users  found  the  new  equipment  and  online  computer-based 
training  to  be  suitable.  Course  content  is  available  on 
computer  disks  for  instances  where  slow  internet  connections 
are  a  problem. 

•  Current  Joint  Program  Manager  -  Information  Systems 
(JPM-IS)  plans  are  not  suitable  for  the  installation, 
configuration,  and  sustainment  of  JWARN  capability  in  the 
complex  Navy  network  operating  environment.  Prior  to 
operational  test,  the  JWARN  software  was  installed  on  the 
ship’s  and  MOC  servers.  Then,  Program  Office  personnel 
were  required  to  configure  JWARN  for  operational  use  and  fix 
problems  at  each  site  prior  to  the  test. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  JWARN 
Program  Office  successfully  addressed  all  previous  FY14 
recommendations . 

•  FY15  Recommendations. 

1.  The  JPM-IS,  in  conjunction  with  the  Program  Management 
Office  for  host  systems,  should  review  plans  for  the 
installation  and  configuration  of  JWARN  and  ensure 
adequate  resources  and  training  are  provided  to  install 

and  sustain  JWARN  in  the  Navy  network  operating 
environments  over  time.  This  review  should  address  a 
process  for  verifying  JWARN  operation  and  interoperability 
at  each  location  after  network  updates  and  software  patches 
and  recurring  deployment  check  outs  to  ensure  operational 
capability. 

2.  The  Navy  should  establish  recurring  training  standards  for 
JWARN  operators  and  CVN  Tactical  Action  Officers  and 
consider  incorporating  JWARN  into  fleet  training  exercises. 

3.  The  Navy  should  conduct  recurring  deployment 
interoperability  verification. 
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Key  Management  Infrastructure  (KMI)  Increment  2 


Executive  Summary 

•  In  coordination  with  the  Key  Management  Infrastructure 
(KMI)  Program  Management  Office  (PMO),  the  Joint 
Interoperability  Test  Command  (JITC)  conducted  a 
Limited  User  Test  (LUT)  of  Spiral  2,  Spin  1  capabilities  in 
April  2015,  and  a  LUT  Retest  in  July  2015  to  verify  deficiency 
corrections.  Testing  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan. 

•  Users  are  satisfied  with  Spiral  2,  Spin  1  capabilities, 
performance,  and  system  stability.  Database  management 
problems  during  the  LUT  and  LUT  Retest  affected  software 
downloading.  Site  failover,  Advance  Extremely  High 
Frequency  keying,  card  loader,  F-22,  KMI  tokens,  benign 
fill,  and  existing  Spiral  1  functions  worked.  During  the  LUT 
Retest,  some  problems  remained  with  Mobile  User  Objective 
System,  Secure  Software  Provisioning,  and  the  Host-Based 
Security  System  (HBSS)  and  its  supporting  servers. 

•  The  National  Security  Agency’s  (NSA)  KMI  Help  Desk 
and  tiered  engineering  support  personnel  lacked  specific 
transition-related  knowledge  and  not  enough  experienced 
personnel  were  available  to  support  extended  coverage  hours. 
NSA  and  Service  Help  Desks  are  not  prepared  for  surge 
transition  and  sustainment. 

•  KMI  Spiral  2,  Spin  2  developmental  and  operational  testing 
is  at  least  12  months  behind  schedule,  and  the  program  will 
probably  not  be  able  to  meet  its  Full  Deployment  Decision  in 
April  2017. 

•  JITC  and  Service  test  participants  identified  a  growing  backlog 
of  high-priority  deficiencies  that  remain  unresolved.  The 
Service  leads  requested  that  the  PMO  resolve  the  backlog  of 
deficiencies  before  continuing  new  development. 

System 

•  KMI  is  intended  to  replace  the  legacy  Electronic  Key 
Management  System  (EKMS)  to  provide  a  means  for  securely 
ordering,  generating,  producing,  distributing,  managing, 

and  auditing  cryptographic  products  (e.g.,  encryption  keys, 
cryptographic  applications,  and  account  management). 

•  KMI  consists  of  core  nodes  that  provide  web  operations 
at  sites  operated  by  the  NSA,  as  well  as  individual  client 
nodes  distributed  globally  to  enable  secure  key  and  software 
provisioning  services  for  the  DOD,  intelligence  community, 
and  agencies. 

•  KMI  combines  substantial  custom  software  and  hardware 
development  with  commercial  off-the-shelf  computer 
components.  The  custom  hardware  includes  an  Advanced 
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Key  Processor  for  autonomous  cryptographic  key  generation 
and  a  Type  1  user  token  for  role-based  user  authentication. 
The  commercial  off-the-shelf  components  include  a  client 
host  computer  with  monitor  and  peripherals,  High  Assurance 
Internet  Protocol  Encryptor  (KG-250),  printer,  and  barcode 
scanner. 

Mission 

•  Combatant  Commands,  Services,  DOD  agencies,  other 
Federal  Government  agencies,  coalition  partners,  and 
allies  will  use  KMI  to  provide  secure  and  interoperable 
cryptographic  key  generation,  distribution,  and  management 
capabilities  to  support  mission-critical  systems,  the  DOD 
Information  Networks,  and  initiatives  such  as  Cryptographic 
Modernization. 

•  Service  members  will  use  KMI  cryptographic  products 
and  services  to  enable  security  services  (confidentiality, 
non-repudiation,  authentication,  and  source  authentication) 
for  diverse  systems  such  as  Identification  Friend  or  Foe,  GPS, 
Advanced  Extremely  High  Frequency  Satellite  System,  and 
Warfighter  Information  Network  -  Tactical. 

Major  Contractors 

•  Leidos  -  Columbia,  Maryland  (Spiral  2  Prime) 

•  General  Dynamics  Information  Assurance 
Division  -  Needham,  Massachusetts  (Spiral  1  Prime) 

•  L3  Communications  -  Camden,  New  Jersey 
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Activity 

•  In  coordination  with  the  KMI  PMO,  JITC  conducted  a  LUT  of 
Spiral  2,  Spin  1  capabilities  in  April  2015,  and  a  LUT  Retest  in 
July  2015  to  verify  deficiency  corrections,  in  accordance  with 
a  DOT&E-approved  test  plan. 

•  Sixty-nine  operationally  representative  Air  Force,  Army, 
Marine  Corps,  Navy,  and  civil  KMI  account  users  participated 
during  the  LUT  and  its  retest  at  geographically  dispersed  sites. 

•  DOT&E  submitted  a  classified  report  detailing  results  of  the 
LUT  and  LUT  Retest  in  October  2015. 

•  NS  A  and  JITC  evaluated  KMI  Spiral  2,  Spin  1  cybersecurity 
in  the  LUT  Retest;  the  results  are  classified. 

•  JITC  is  developing  plans  for  a  Spiral  2,  Spin  2  Operational 
Assessment  in  2QFY16  and  a  LUT  to  be  conducted  in 
4QFY16. 

Assessment 

•  Users  are  satisfied  with  Spiral  2,  Spin  1  capabilities, 
performance,  and  system  stability.  Functionality  improved  for 
the  LUT  Retest  but  suitability  problems  remained. 

•  Database  management  problems  during  the  LUT  and  LUT 
Retest  affected  software  downloading.  Site  failover,  Advanced 
Extremely  High  Frequency  keying,  card  loader,  F-22,  KMI 
tokens,  benign  fill,  and  existing  Spiral  1  functions  worked. 
During  the  LUT  Retest,  some  problems  remained  with  Mobile 
User  Objective  System,  Secure  Software  Provisioning,  and  the 
HBSS  and  its  supporting  server. 

•  The  PMO’s  KMI  token  reliability  growth  program  continues 
to  identify  fault  modes  and  has  demonstrated  improved 
reliability. 

•  KMI  Spiral  2,  Spin  2  developmental  and  operational  testing 
is  at  least  12  months  behind  schedule,  and  the  program  will 
probably  not  meet  its  Full  Deployment  Decision  in  April  2017. 

•  JITC  and  Service  test  participants  identified  a  growing  backlog 
of  high-priority  deficiencies  that  remain  unresolved.  The 
Service  leads  requested  that  the  PMO  resolve  the  backlog  of 
deficiencies  before  continuing  new  development. 

•  The  LUT  Retest  concluded  with  only  one  high-priority 
product  inventory  discrepancy. 

•  The  KMI  program  implemented  a  re-verification  process 
for  account  holders,  Advanced  Key  Processor,  tokens,  and 
the  client  that  creates  unannounced  service  interruptions 
when  re-verifications  are  missed.  The  re-verifications  and 
HBSS-enforced  software  version  controls  prevent  KMI 
from  operating  autonomously  for  6-9  months  as  designed. 
NSAmust  address  these  process-related  system-enforced 
conflicts,  to  enable  the  National  Guard,  Army  Reserve, 
remotely-deployed  units,  and  submarine  forces  to  be  able  to 
operate  with  KMI. 

•  During  the  LUT,  the  Army  identified  26  new  KMI  tokens 
at  the  depot  that  failed  at  initialization  out-of-the  box 
(10.4  percent  failure  rate),  indicating  problems  with  the 
manufacturing  production  process.  The  PMO  corrected 


problems  in  manufacturing,  which  helped  bring  the  overall 
depot  failure  rate  for  both  the  LUT  and  LUT  Retest  down  to 
2.6  percent  (52  out  of  1,978  tokens). 

•  The  KMI  PMO  successfully  demonstrated  continuity  of 
operations  planning  and  execution,  by  conducting  a  failover  to 
the  backup  site  during  live  operations. 

•  The  NSA  Help  Desk  and  tiered  engineering  support  personnel 
lacked  specific  transition-related  knowledge.  In  addition, 
not  enough  experienced  KMI  engineering,  second  echelon, 
system  administrators,  database  management,  and  Help 
Desk  personnel  were  available  to  support  extended  coverage 
hours;  this  problem  was  previously  reported  by  DOT&E  at 
the  2012  KMI  IOT&E,  2013  FOT&E,  and  again  at  the  2014 
Operational  Assessment.  The  NSA  and  Service  Help  Desks 
are  not  prepared  for  surge  transition  and  sustainment,  as  some 
new  Help  Desk  technicians  lack  KMI  experience  and  system 
knowledge.  This  was  especially  noticeable  during  transition 
support. 

•  Problems  observed  in  previous  testing,  if  not  corrected  during 
system  development,  could  adversely  affect  the  system’s 
effectiveness,  suitability,  or  survivability  during  the  KMI 
Spiral  2,  Spin  2  LUT,  which  is  scheduled  to  begin  in  4QFY16. 

-  There  may  be  latent  software  flaws  that  could  affect 
ongoing  mission  operations. 

-  NSA  and  Service  Help  Desk  manning  may  not  be 
adequately  staffed  to  support  the  pace  of  transition  from 
EKMS  to  KMI. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  KMI  PMO 
satisfactorily  addressed  two  of  the  five  FY14 
recommendations.  The  following  remain  unresolved: 

1 .  Continue  to  improve  rigor  of  the  KMI  software 
development  and  regression  process  to  identify  and  resolve 
problems  before  entering  operational  test  events. 

2.  Allot  adequate  schedule  time  to  support  test  preparation, 
regression,  post-test  data  analysis,  verification  of 
corrections,  and  reporting  to  support  future  deployment  and 
fielding  decisions. 

3.  Continue  to  verify  increased  KMI  token  reliability  through 
a  combination  of  laboratory  and  operational  testing  with 
automated  data  collection  from  system  logs  for  accurate 
reliability  and  usage  analysis. 

•  FY15  Recommendations.  The  KMI  PMO  should: 

1 .  Resolve  the  mounting  backlog  of  deficiencies  and  establish 
a  regular  maintenance  release  schedule. 

2.  Ensure  that  appropriate  transition  and  funding  plans  are 
in-place  to  continue  development  and  support  fielding 
efforts  beyond  FY 1 7  target  dates. 

3.  Resolve  HBSS  version  management  and  re-verification 
process  problems  that  obstruct  autonomous  operations. 
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4.  Implement  improved  and  rigorous  configuration 
management.  Configuration  Control  Board,  Information 
Assurance  Vulnerability  Alert  update  processes  and  controls 
to  properly  sustain  KMI. 

5.  Ensure  adequate  engineering,  second  echelon,  system 
administrators,  database  managers,  and  NSA  and  Service 
Help  Desk  and  transition  staff  are  available  to  support  surge 
fielding  and  long-term  KMI  sustainment. 
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Public  Key  Infrastructure  (PKI)  Increment  2 


CCEB  -  Combined  Communications  Electronic  Board  RA  -  Registration  Authority 

DM  DC  -  Defense  Manpower  Data  Center  RAPIDS  -  Real-Time  Automated  Personnel  Identification  System 

DODIN  -  Department  of  Defense  Information  Networks  RCVS  -  Robust  Certificate  Validation  Service 

ECA  -  External  Certification  Authority  SIPRNET  -  SECRET  Internet  Protocol  Router  Network 

LRA  -  Local  Registration  Authority 


Executive  Summary 

•  FOT&Es  I  and  II,  conducted  in 
January  2013,  revealed  effectiveness 
and  suitability  problems.  Although  no 
operational  testing  has  been  completed 
since  then,  the  program  manager  is 
addressing  the  requirements  definition 
and  system  engineering  problems 
that  led  to  these  deficiencies,  while 
also  making  program  personnel  and 
contract  management  process  changes  to 
improve  the  program’s  ability  to  achieve 
restructured  goals. 

•  In  December  2014,  the  Public  Key 
Infrastructure  (PKI)  Increment  2  Program 
Management  Office  (PMO)  and  the  Joint 
Interoperability  Test  Command  (JITC) 
conducted  an  integrated  developmental 
test/operational  test  (DT/OT)  for  Token 
Management  System  (TMS)  release  3.0 
to  examine  code  signing,  token  Personal 
Identification  Number  (PIN)  reset, 
certificate  recovery,  and  additional  token 
issuance  capabilities. 

•  USD(AT&L),  guided  by  the  recommendation  of  the  Director 
of  the  National  Security  Agency  (NSA),  directed  the  PKI 
PMO  to  evaluate  the  viability  of  whether  a  one-token  and/or 
one-infrastructure  approach  could  achieve  the  DOD  PKI 
mission  requirements  for  both  the  classified  and  unclassified 
networks.  This  resulted  in  a  program  delay  of  six  months. 

•  USD(AT&L)  signed  an  Acquisition  Decision  Memorandum 
(ADM)  in  April  2015,  which  concluded  that  a  one 

token/ one-infrastructure  approach  would  cost  more  money, 
take  longer  to  develop,  and  would  not  improve  cybersecurity. 
The  ADM,  in  conjunction  with  the  Joint  Requirements 
Oversight  Council  memorandum,  directed  the  NSA  to  resume 
development  of  the  PKI  Increment  2  program  in  accordance 
with  plans  developed  prior  to  the  strategic  pause. 

•  In  April  and  May  2015,  JITC  verified  correction  of 
deficiencies  to  resolve  problems  found  in  the  integrated 
DT/OT  for  the  TMS  release  3.0. 

•  In  July  2015,  USD(AT&L)  approved  the  revised  PKI 
Acquisition  Program  Baseline,  and  approved  the  updated  PKI 
Acquisition  Strategy  in  September  2015  outlining  the  PKI 
PMO’s  plans  to  complete  Spirals  3  and  4  by  2QFY18.  The 
revised  strategy  focuses  the  remaining  Increment  2  Spirals 

(3  and  4)  on  15  user-prioritized  capabilities.  These  capabilities 
are  intended  to  improve  the  Secret  Internet  Protocol  Router 
Network  (SIPRNET)  token  management  and  reporting, 
improve  system  availability,  and  provide  new  infrastructures 
for  the  provisioning  and  management  of  the  Non-classified 


Internet  Protocol  Router  Network  (NIPRNET)  Enterprise 
Alternate  Token  System  and  Non-Person  Entity  certificates 
(e.g.,  workstations,  web  servers,  and  mobile  devices). 

System 

•  DOD  PKI  provides  for  the  generation,  production, 
distribution,  control,  revocation,  recovery,  and  tracking  of 
public  key  certificates  and  their  corresponding  private  keys. 
The  private  keys  are  encoded  on  a  token,  which  is  a  credit  card 
sized  smartcard  embedded  with  a  microchip. 

•  DOD  PKI  supports  the  secure  flow  of  information  across  the 
DOD  Information  Networks  as  well  as  secure  local  storage  of 
information. 

•  DOD  PKI  uses  commercial  off-the-shelf  hardware,  software, 
and  applications  developed  by  the  NSA. 

-  The  Defense  Enrollment  Eligibility  Reporting  System 
(DEERS)  and  Secure  DEERS  provide  the  personnel  data 
for  certificates  imprinted  on  NIPRNET  Common  Access 
Cards  and  SIPRNET  tokens,  respectively. 

-  DOD  PKI  Certification  Authorities  for  the  NIPRNET 
and  SIPRNET  tokens  reside  in  the  Defense  Information 
Systems  Agency  Enterprise  Service  Centers  in  Oklahoma 
City,  Oklahoma,  and  Mechanicsburg,  Pennsylvania. 

•  Increment  1  is  complete  and  deployed  on  the  NIPRNET. 

•  The  NSA  is  developing  and  deploying  PKI  Increment  2 
in  four  spirals  on  the  SIPRNET  and  NIPRNET.  Spirals  1 
and  2  are  deployed,  while  Spirals  3  and  4  will  deliver  the 
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infrastructure,  PKI  services  and  products,  and  logistical 
support  required  by  15  user-prioritized  capabilities. 

•  The  Defense  Information  Systems  Agency  is  supporting  PKI 
operations,  enablement,  and  security  solutions. 

Mission 

•  Military  operators,  communities  of  interest,  and  other 
authorized  users  will  use  DOD  PKI  to  securely  access, 
process,  store,  transport,  and  use  information,  applications, 
and  networks. 

•  Commanders  at  all  levels  will  use  DOD  PKI  to  provide 
authenticated  identity  management  via  personal  identification, 
number-protected  Common  Access  Cards  or  S1PRNET  tokens 


to  enable  DOD  members,  coalition  partners,  and  others  to 
access  restricted  websites,  enroll  in  online  services,  and 
encrypt  and  digitally  sign  e-mail. 

•  Military  network  operators  will  use  Non-Person  Entity 
certificates  (e.g.,  workstations,  web  servers,  and  mobile 
devices)  to  create  secure  network  domains,  which  will 
facilitate  intrusion  protection  and  detection. 

Major  Contractors 

•  General  Dynamics  C4  Systems  -  Scottsdale,  Arizona  (Prime) 

•  90Meter  -  Newport  Beach,  California 

•  SafeNet  -  Belcamp,  Maryland 


Activity 

•  In  December  2014,  the  PKI  PMO  and  JITC  conducted  an 
integrated  DT/OT  for  TMS  release  3.0  to  examine  code 
signing,  token  PIN  reset,  certificate  recovery,  and  additional 
token  issuance  capabilities. 

•  USD(AT&L),  guided  by  the  recommendation  of  the  Director 
of  NSA,  directed  the  PKI  Increment  2  PMO  to  evaluate  the 
viability  of  whether  a  one-token  and/or  one-infrastructure 
approach  could  achieve  the  DOD  PKI  mission  requirements 
for  both  the  classified  and  unclassified  networks.  This  resulted 
in  a  program  delay  of  six  months. 

•  USD(AT&L)  signed  an  ADM  in  April  2015,  which  concluded 
that  a  one-token/one-infrastructure  approach  would  cost 
more  money,  take  longer  to  develop,  and  would  not  improve 
cybersecurity.  The  ADM,  in  conjunction  with  the  Joint 
Requirements  Oversight  Council  memorandum,  directed  the 
NSA  to  resume  development  of  the  PKI  Increment  2  Program 
in  accordance  with  previous  plans  developed  prior  to  the 
strategic  pause. 

•  In  April  and  May  2015,  JITC  conducted  a  correction  of 
deficiency  verification  test  to  resolve  problems  found  in  the 
integrated  DT/OT  for  the  TMS  release  3.0. 

•  USD(AT&L)  convened  an  Integrated  Product  Team  to  evaluate 
TMS  release  3.0  in  June  2015  and  issued  a  fielding  ADM  in 
September  2015. 

•  USD(AT&L)  approved  the  revised  PKI  Acquisition  Program 
Baseline  in  July  2015,  and  approved  the  updated  PKI 
Acquisition  Strategy  in  September  2015  outlining  the  PKI 
PMO’s  plans  to  complete  Spirals  3  and  4  by  2QFY18.  The 
revised  strategy  focuses  the  remaining  Increment  2  Spirals 

(3  and  4)  on  15  user-prioritized  capabilities.  These  capabilities 
are  intended  to  improve  the  SIPRNET  token  management 
and  reporting,  improve  system  availability,  and  provide  new 
infrastructures  for  the  provisioning  and  management  of  the 
NIPRNET  Enterprise  Alternate  Token  System  and  Non-Person 
Entity  certificates. 

•  The  PMO  and  test  community  are  finalizing  the  Spiral  3 
Test  and  Evaluation  Master  Plan  (TEMP)  Addendum  in 
November  2015  for  signature  approval  in  January  2016. 


•  The  PMO  is  also  updating  the  PKI  System  Engineering  Plan, 
Life  Cycle  Sustainment  Plan,  and  Transition  Plan. 

•  The  PMO  had  no  major  operational  test  events  scheduled  in 
FY15,  but  does  have  test  events  scheduled  for  2QFY16  for 
TMS  4.1,  4.2,  and  4.3  capabilities. 

Assessment 

•  Delaying  the  capability  deliveries  until  FY16,  which  were 
due  to  the  six-month  one-token/one-infrastructure  strategic 
pause,  negatively  affected  developmental  and  operational  test 
planning  and  execution.  TMS  release  3.0  is  a  minor  FY15 
release  that  will  not  change  the  overall  not  effective/not 
suitable  evaluations. 

•  FOT&Es  1  and  II,  conducted  in  January  2013,  revealed 
effectiveness  and  suitability  problems.  Although  no 
operational  testing  has  been  completed  since  then,  the  program 
manager  is  addressing  the  requirements  definition  and  system 
engineering  problems  that  led  to  these  deficiencies,  while 
also  making  program  personnel  and  contract  management 
process  changes  to  improve  the  program’s  ability  to  achieve 
restructured  goals. 

•  The  PKI  PMO’s  and  JITC’s  integrated  DT/OT,  combined  with 
the  subsequent  correction  of  deficiencies  verification  of  TMS 
release  3.0  capabilities  in  April  and  May  2015,  confirmed  that 
Spiral  3  is  adequate  for  DOD-wide  fielding.  USD(AT&L) 
issued  a  fielding  ADM  in  September  2015  to  authorize  the  use 
of  the  new  capabilities  including  token  PIN  resets,  encryption 
certificate  key  recovery,  and  issuance  of  new  token  types 
(Unique  Identification-based  and  Administrator  Identity-only 
certificates). 

•  The  PKI  PMO’s  methodology  for  measuring  token  reliability 
lacks  sufficient  rigor  and  focus.  The  PMO  should  focus  their 
long-term  reliability  growth  plan  and  testing  on  the  end-state 
token.  Furthermore,  the  current  token  reliability  requirement 
requires  6,000  hours  over  3  years,  which  equates  to  35  hours  a 
week.  Many  DOD  users  require  a  higher  reliability.  The  DOD 
Chief  Information  Officer  directed  the  PKI  PMO  to  define  a 
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higher  reliability  requirement  for  the  tokens  in  August  2014 
that  remains  unresolved. 

•  DOT&E  conducted  a  Spiral  3  TEMP  Addendum  early  review 
in  September  2015,  and  the  document  is  on  track  for  approval 
in  January  2016. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  PKI  PMO 
satisfactorily  addressed  the  three  previous  FY14 
recommendations . 

•  FY15  Recommendations.  The  PKI  PMO  should: 

1.  Develop  the  Spiral  4  TEMP  Addendum  in  accordance  with 
the  redefined  PKI  Increment  2  Acquisition  Strategy  to 


prepare  stakeholders  for  the  remaining  deliveries,  resource 
commitments,  and  T&E  goals. 

2.  Create  a  Spiral  4  transition  plan  defining  roles  and 
responsibilities  for  stakeholders  to  support  a  smooth 
transition  and  ensure  minimal  impact  to  PKI  operations 
once  the  program  enters  sustainment. 

3.  Define  and  validate  sustainment  requirements  for  PKI 
Spiral  4  capabilities. 

4.  Provide  periodic  reports  of  token  reliability,  failure  rates, 
and  root  cause  analyses. 
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Theater  Medical  Information  Program  -  Joint  (TMIP-J) 


Executive  Summary 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  conducted 
a  Multi-Service  Operational  Test  and  Evaluation  (MOT&E) 
of  Theater  Medical  Information  Program  -  Joint  (TMIP-J) 
Increment  2  Release  3,  from  August  13  -  21,  2015.  The  Air 
Force  Operational  Test  and  Evaluation  Center,  Marine  Corps 
Operational  Test  and  Evaluation  Activity,  United  States  Army 
Medical  Department  Board,  the  Air  Force  Medical  Evaluation 
Support  Activity  (AFMESA),  Marine  Corps  Tactical  Systems 
Support  Activity  (MCTSSA),  and  the  Joint  Interoperability 
Test  Command  (JITC)  all  participated  in  the  MOT&E. 

•  The  Navy  initiated  its  portion  of  the  MOT&E  aboard  the 
USS  Carter  Hall  (LSD-50)  on  November  2,  2015.  The  Navy 
Information  Operations  Command  (NIOC)  -  Norfolk  Red 
Team  will  conduct  cybersecurity  assessments  aboard  the  USS 
Carter  Hall  (LSD-50)  from  January  7  -  11,  2016. 

•  The  Army,  Air  Force,  and  Marine  Corps  completed  MOT&E 
data  collection  at  the  end  of  FY15,  and  DOT&E  began 
evaluation  of  the  test  data  in  early  FY16. 

System 

•  TMIP-J  is  a  Major  Automated  Information  System  that 
integrates  software  from  sustaining  base  medical  applications 
into  a  multi-Service  system  for  use  by  deployed  forces. 
Examples  of  integrated  applications  include  the  theater 
versions  of  the  Armed  Forces  Health  Longitudinal  Technology 
Application,  Composite  Health  Care  System,  and  Defense 
Medical  Logistics  Standard  Support. 

•  TMIP-J  provides  the  following  medical  capabilities: 

-  Electronic  Health  Records 

-  Medical  command  and  control 

-  Medical  logistics 

-  Patient  movement  and  tracking 

-  Patient  data  to  populate  the  Theater  Medical  Data  Store 
(theater  database)  and  the  Clinical  Data  Repository 
(Continental  U.S.  database) 

•  The  Services  provide  their  own  infrastructure  (networks  and 
communications)  and  computer  hardware  to  host  the  TMIP-J 
software. 

•  TMIP-J  consists  of  two  increments.  The  Program  Executive 
Office  fielded  Increment  1  in  2003  and  is  developing 
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Increment  2  in  multiple  releases  with  the  following  fielding 
dates: 

-  Increment  2  Release  1  -  fielded  in  2009. 

-  Increment  2  Release  2  -  fielded  in  2014. 

-  Increment  2  Release  3  was  the  system  under  test  during 
2015  and  is  the  final  TMIP-J  release. 

•  The  Program  Executive  Office  initiated  the  Joint  Operational 
Medicine  Information  Systems  program  in  FY15.  This 
program  will  replace  portions  of  TMIP-J. 

Mission 

•  Combatant  Commanders,  Joint  Task  Force  commanders,  and 
their  medical  staff  equipped  with  TMIP-J  can  make  informed 
and  timely  decisions  about  planning  and  delivering  health  care 
services  in  the  theater. 

•  Military  health  care  providers  equipped  with  TMIP-J  can 
electronically  document  medical  care  provided  to  deployed 
forces  to  support  continuity  of  medical  care  from  the  theater  to 
the  sustaining  base. 

Major  Contractors 

•  SAIC  -  Falls  Church,  Virginia 

•  Northrop  Grumman  -  Chantilly,  Virginia 

•  Akimeka  LLC,  Kihei  -  Maui,  Hawaii 


Activity 

•  ATEC  conducted  an  MOT&E  of  TMIP-J  Increment  2 

Release  3,  in  accordance  with  the  DOT&E-approved  test  plan, 
from  August  13  -  21,  2015.  The  Air  Force  Operational  Test 
and  Evaluation  Center,  Marine  Corps  Operational  Test  and 
Evaluation  Activity,  United  States  Army  Medical  Department 
Board,  AFMESA,  MCTSSA,  and  JITC  also  participated  in  the 


MOT&E.  ATEC  tested  the  Army  and  Air  Force  components 
of  TMIP-J  at  AFMESA,  Fort  Detrick,  Maryland,  and  Marine 
Corps  portions  of  TMIP-J  at  MCTSSA,  Camp  Pendleton, 
California. 

•  In  January  2015,  the  Army  Research  Laboratory  Survivability/ 
Lethal  Analysis  Directorate  conducted  a  Cooperative 
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Vulnerability  and  Penetration  cybersecurity  assessment  of  the 
Army,  Marine  Corps,  and  Air  Force  portions  of  TMIP-J. 

•  In  August  2015,  the  Threat  System  Management  Office 
conducted  a  cybersecurity  Adversarial  Assessment  for  the 
Army,  Air  Force,  and  Marine  Corps  portions  of  TMIP-J  in 
conjunction  with  the  MOT&E. 

•  JITC  reviewed  all  system  interfaces  and  identified  three 
joint  interfaces  that  require  interface  testing  to  complete 
the  TMIP-J  Interoperability  Certification  process.  JITC 
conducted  interface  testing  in  conjunction  with  the  MOT&E 
in  August  2015  and  is  collecting  additional  data  in  conjunction 
with  the  Navy  portion  of  the  MOT&E. 

•  The  Navy  initiated  its  portion  of  the  MOT&E  aboard  the 
USS  Carter  Hall  (LSD-50)  on  November  2,  2015.  The 
NIOC  -  Norfolk  Red  Team  will  conduct  cybersecurity 
assessments  aboard  the  USS  Carter  Hall  (LSD-50) 
from  January  7—11,  2016. 


•  DOT&E  will  submit  a  TMIP-J  MOT&E  report  for  the  Army, 
Air  Force,  and  Marine  Corps  portions  of  TMIP-J  and  an 
addendum  report  to  the  Navy  in  2QFY16. 

Assessment 

•  The  Army,  Air  Force,  and  Marine  Corps  completed  data 
collection  at  the  end  of  FY15,  and  DOT&E  began  evaluation 
of  the  test  data  in  early  FY16. 

•  DOT&E ’s  MOT&E  report  will  detail  the  results  of  testing  on 
the  Army,  Air  Force,  and  Marine  Corps  portions  of  TMIP-J. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program 
satisfactorily  addressed  all  previous  FY13  recommendations 
except  for  interface  testing.  JITC  plans  to  complete  interface 
testing  in  1QFY16. 

•  FY15  Recommendations.  None. 
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Network  Integration  Evaluation  (NIE) 


In  FY15,  the  Army  conducted  two  Network  Integration 
Evaluations  (NIEs)  at  Fort  Bliss,  Texas,  and  White  Sands  Missile 
Range,  New  Mexico.  NIE  15.1  was  conducted  in  October  and 
November  2014,  and  NIE  15.2  was  conducted  in  April  and 
May  2015.  The  purpose  of  the  NIEs  is  to  provide  a  venue 
for  operational  testing  of  Army  acquisition  programs,  with  a 
particular  focus  on  the  integrated  testing  of  tactical  mission 
command  networks.  The  Army  also  intends  the  NIEs  to  serve  as 
a  venue  for  evaluating  emerging  capabilities  that  are  not  formal 
acquisition  programs.  These  systems,  termed  by  the  Army 
as  “systems  under  evaluation,”  are  not  acquisition  programs 
of  record,  but  rather  systems  that  may  offer  value  for  future 
development. 

The  Army’s  intended  objective  of  the  NIE  to  test  and  evaluate 
network  components  in  a  combined  event  is  sound.  The  NIE 
events  should  allow  for  a  more  comprehensive  evaluation  of 
an  integrated  mission  command  network,  instead  of  piecemeal 
evaluations  of  individual  network  components. 

NIE  15.1 

During  NIE  15.1,  the  Army  conducted  an  FOT&E  for  Warfighter 
Information  Network  -  Tactical  (WIN-T)  Increment  2.  An 
individual  article  providing  an  assessment  of  WIN-T  Increment  2 
can  be  found  separately  in  this  annual  report. 


NIE  15.2 

During  NIE  15.2,  the  Army  conducted  an  FOT&E  for  the 
Distributed  Common  Ground  System  -  Army  and  a  Limited  User 
Test  for  the  Mid-Tier  Networking  Radio.  Individual  articles  on 
these  programs  are  provided  elsewhere  in  this  annual  report. 


NIE  ASSESSMENT 


NIE  15.1  and  15.2  were  the  eighth  and  ninth  such  events 
conducted  to  date.  The  Army  has  developed  a  systematic 
approach  to  preparing  for  and  conducting  NIEs  and  applying 
lessons  learned  from  previous  events.  Overall,  NIEs  have  been  a 
satisfactory  venue  for  conducting  operational  tests  of  individual 
network  acquisition  programs. 

Operational  Scenarios  and  Test  Design.  The  Army  Test 
and  Evaluation  Command’s  Operational  Test  Command,  in 
conjunction  with  the  Brigade  Modernization  Command,  continue 
to  develop  realistic,  well-designed  operational  scenarios  for 
use  during  NIEs.  Additionally,  the  2d  Brigade,  1st  Armored 
Division,  as  a  dedicated  NIE  test  unit,  is  a  valuable  resource  for 
the  conduct  of  NIEs. 

Future  NIEs  should  continue  to  develop  new  and  more 
demanding  operational  scenarios  to  reflect  future  combat 
operations.  Future  NIEs  should  include  challenging  and  stressful 
combined  arms  maneuvers  against  regular  conventional  forces. 
Such  scenarios  would  place  greater  stress  on  the  tactical  network 
and  elicit  a  more  complete  assessment  of  that  network.  Within 
resource  constraints,  the  Army  should  continue  to  strive  to  create 
a  demanding  operational  environment  at  NIEs  similar  to  that 
found  at  the  Army’s  combat  training  centers. 


Testing  and  Experimentation.  Beginning  in  FY16,  the  Army 
will  devote  one  NIE  a  year  to  operational  testing  and  another 
annual  event  to  experimentation  and  force  development.  The 
latter  event  is  to  be  called  an  Army  Warfighting  Assessment, 
the  first  of  which  will  be  conducted  in  October  2015.  This  new 
approach  is  intended  to  pay  dividends  by  focusing  individual 
event  design  on  the  specific  requirements  of  either  testing  or 
experimentation. 

Instrumentation  and  Data  Collection.  The  Army  should 
continue  to  improve  its  instrumentation  and  data  collection 
procedures  to  support  operational  testing.  For  example,  the 
Army  Test  and  Evaluation  Command  should  devote  effort 
towards  developing  instrumentation  to  collect  network  data  for 
dismounted  radios,  such  as  the  Manpack  radio.  Additionally,  the 
Army  needs  to  place  greater  emphasis  on  the  use  of  Real-Time 
Casualty  Assessment  instrumentation,  which  is  an  essential 
component  of  good  force-on-force  operational  testing,  such  as 
that  conducted  at  NIEs.  A  Real-Time  Casualty  Assessment  is 
intended  to  accurately  simulate  direct  and  indirect  fire  effects 
for  both  friendly  and  threat  forces.  Finally,  the  Army  should 
continue  to  refine  its  methodology  for  the  conduct  of  interviews, 
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focus  groups,  and  surveys  with  the  units  employing  the  systems 
under  test. 

Threat  Operations.  An  aggressive,  adaptive  threat  intent  on 
winning  the  battle  is  an  essential  component  of  good  operational 
testing.  The  Army  continues  to  improve  threat  operations  during 
NIEs,  particularly  with  respect  to  threat  information  operations, 


such  as  electronic  warfare  and  computer  network  operations. 
NIEs  should  incorporate  a  large,  challenging  regular  force  threat 
that  includes  a  sizeable  armored  force  and  significant  indirect  fire 
capabilities.  Threat  capabilities  should  be  upgraded  each  year  to 
reflect  real-world  threat  developments. 


NETWORK  PERFORMANCE  OBSERVATIONS 


The  following  are  observations  of  tactical  network  performance 
during  NIEs.  These  observations  focus  on  network  performance 
deficiencies  that  the  Army  should  consider  as  it  moves  forward 
with  integrated  network  development. 

Complexity  of  Use.  Network  components,  both  mission 
command  systems  and  elements  of  the  transport  layer,  remain 
excessively  complex  to  use.  The  current  capability  of  an 
integrated  network  to  enhance  mission  command  is  diminished 
due  to  pervasive  task  complexity.  It  is  challenging  to  achieve  and 
maintain  user  proficiency. 

Networking  Waveforms.  The  Army  is  committed  to  using 
networking  waveforms,  such  as  the  Soldier  Radio  Waveform 
and  Wideband  Networking  Waveform,  to  implement  a 
networked  tactical  communications  network.  While  networked 
communications  at  lower  tactical  levels  may  create  enhanced 
operational  capability,  the  use  of  these  networking  waveforms 
brings  negative  attributes,  which  need  to  be  fully  evaluated 
and  understood.  For  example,  these  waveforms,  due  to  their 
higher  frequencies,  have  shorter  ranges  and  are  more  affected 
by  terrain  obstructions  compared  to  the  legacy  Single  Channel 
Ground  and  Airborne  Radio  System  waveform.  Establishing 
and  maintaining  networked  communications  is  complex  and 
difficult.  For  example,  loading  the  initial  network  plans  in  all 
the  necessary  radios,  updating  the  network  to  accommodate  a 
new  unit  task  organization,  and  conducting  a  communications 
security  changeover  are  lengthy  and  cumbersome  tasks  requiring 
each  individual  radio  to  be  manually  updated.  This  process 
typically  requires  in  excess  of  24  hours  for  a  Brigade  Combat 
Team  to  complete;  this  is  an  excessive  length  of  time  for  a  unit 
conducting  combat  operations.  Networked  radios  also  have  a 
much  higher  power  consumption  resulting  in  significantly  higher 
battery  consumption  rates  for  dismounted  radios.  Finally,  since 
networked  communications  are  constantly  emitting,  they  are 
much  more  vulnerable  to  threat  electronic  direction  finding. 

Armored  Brigade  Combat  Team  Integration.  The  challenge 
of  integrating  network  components  into  tracked  combat  vehicles 
remains  unresolved.  Due  to  vehicle  space  and  power  constraints, 
the  Army  has  yet  to  successfully  integrate  desired  network 
capabilities  into  Abrams  tanks  and  Bradley  infantry  fighting 
vehicles.  It  is  not  clear  how  the  desired  tactical  network  will  be 
incorporated  into  heavy  brigades. 

Stryker  Brigade  Combat  Team  Integration.  The  WIN-T 
FOT&E  conducted  during  NIE  15.1  revealed  significant  issues 


with  the  integration  of  WIN-T  into  Stryker  vehicles.  In  both 
the  Stryker  Point  of  Presence  vehicle  and  the  Stryker  Soldier 
Network  Extension  vehicle,  WIN-T  components  were  poorly 
integrated  from  a  human  factors  perspective.  The  placement  of 
these  components  in  the  vehicles  interfered  with  Stryker  crew 
operations  and  negatively  affected  unit  mission  execution. 

Infantry  Brigade  Combat  Team  Integration.  Integration  of 
the  tactical  network  into  an  Infantry  Brigade  Combat  Team  has 
not  been  evaluated  at  NIEs  due  to  the  lack  of  a  light  infantry  unit 
assigned  to  the  NIE  test  unit.  Integration  of  the  network  into 
the  light  forces  will  be  challenging  given  the  limited  number  of 
vehicles  in  the  Infantry  Brigade  Combat  Team.  Most  of  the  key 
network  components,  such  as  Joint  Battle  Command  -  Platform, 
are  hosted  on  vehicles.  The  challenge  of  linking  into  the  tactical 
network  is  particularly  acute  at  company  level  and  below,  where 
light  infantry  units  operate  dismounted. 

Spectrum  Management.  The  intended  tactical  network  places 
a  greater  demand  upon  the  available  electromagnetic  spectrum 
than  has  been  the  case  with  non-networked  communications.  The 
network  topology  requires  more  discrete  frequencies,  which  will 
place  greater  stress  on  a  tactical  unit’s  capability  to  allocate  and 
manage  the  available  spectrum.  This  challenge  will  be  even  more 
significant  for  large  tactical  units,  such  as  divisions,  operating  in 
the  same  geographical  area  of  operations. 

Survivability.  An  integrated  tactical  network  introduces  new 
vulnerabilities  to  threat  countermeasures,  such  as  threat  computer 
network  attacks,  and  the  ability  of  a  threat  to  covertly  track 
friendly  operations.  The  Army  should  continue  to  improve 
its  capability  to  secure  and  defend  its  tactical  network.  The 
Army  should  ensure  that  brigade-level  cybersecurity  teams  are 
appropriately  manned  and  trained. 

Air-Ground  Communications.  The  Army  has  yet  to  integrate 
radios  into  its  rotary-winged  aircraft  which  are  capable  of 
operating  in  the  same  network  as  ground  forces  at  the  company 
level  and  below.  This  remains  an  important  operational  gap. 

Dependence  on  Field  Service  Representatives.  Units  remain 
overly  dependent  upon  civilian  Field  Service  Representatives  to 
establish  and  maintain  the  integrated  network.  This  dependency 
corresponds  directly  to  the  excessive  complexity  of  use  of 
network  components. 


104  NIE 


F  Y  1  5  ARMY  PROGRAMS 


Abrams  M1A2  System  Enhancement  Program  Version  3 

(SEPv3)  Main  Battle  Tank  (MBT) 


Executive  Summary 

•  DOT&E  approved  the  Abrams  M1A2  System  Enhancement 
Program  Version  3  (SEPv3)  Main  Battle  Tank  (MBT) 

Test  and  Evaluation  Master  Plan  and  LFT&E  strategy  on 
March  26,  2015. 

•  In  FY15,  the  Army  continued  testing  to  characterize  the 
performance  of  the  M1A2  SEPv3  Next  Evolutionary  Armor 
(NEA)  against  multiple,  operationally  realistic  threats. 

•  The  Army  conducted  underbody  1ED  testing  against  Ml  A2 
ballistic  hull  and  turrets  (BF1&T)  to  challenge  vulnerability 
reduction  measures  taken  to  improve  the  protection  provided 
by  the  tank  against  underbody  IEDs.  The  Program  Office 
used  these  test  results  to  determine  which  design  changes  to 
integrate  into  the  M1A2  SEPv3  to  improve  underbody  IED 
protection. 

•  DOT&E  is  working  with  the  Army  to  utilize  data  from 
ongoing  test  phases  to  support  its  final  assessment  of  M1A2 
SEPv3  survivability  against  existing  and  emerging  threats  in 
FY20. 

System 

•  The  Abrams  M1A2  SEPv3  is  an  upgrade  to  the  U.S.  Army’s 
current  MBT.  The  M1A2  SEPv3  is  a  tracked,  land  combat, 
assault  weapon  system  possessing  significant  survivability, 
shoot-on-the-move  firepower,  joint  interoperability  (for  the 
exchange  of  tactical  and  support  information),  and  a  high 
degree  of  maneuverability  and  tactical  agility. 

•  The  Army  intends  the  Ml  A2  SEPv3  to  enable  the  crew  to 
engage  the  full  spectrum  of  enemy  ground  targets  with  a 
variety  of  accurate  point  and  area  fire  weapons  in  urban  and 
open  terrain. 

•  The  M1A2  SEPv3  includes  multiple  upgrades  to  improve: 

-  Power  generation  and  distribution  to  support  power 
demands  of  future  technologies 

-  Network  compatibility 

-  Survivability  against  multiple  threats  by  incorporating 
NEA,  a  new  underbody  IED  kit  and  other  vulnerability 
reduction  measures  to  reduce  the  tank’s  vulnerability  to 
IEDs.  These  measures  include  redesigned  crew  seating, 
additional  floor  stiffners,  hardware  to  provide  lower  limb 
protection,  and  changes  in  the  material  and  dimensions  of 
internal  structural  supports. 


-  Lethality  by  providing  the  ability  for  the  fire  control 
system  to  digitally  communicate  with  the  new  large  caliber 
ammunition 

-  Energy  efficiency  (sustainment)  due  to  the  incorporation  of 
an  auxiliary  power  unit 

Mission 

•  Units  equipped  with  the  M1A2  SEPv3  enable  Army  combined 
arms  teams  to  close  with  and  destroy  the  enemy  by  fire  and 
movement  across  the  full  range  of  military  operations. 

•  The  Army  intends  the  M1A2  SEPv3  to  defeat  and/or  suppress 
enemy  tanks,  reconnaissance  vehicles,  infantry  fighting 
vehicles,  armored  personnel  carriers,  anti-tank  guns,  guided 
missile  launchers  (ground  and  vehicle  mounted),  bunkers, 
dismounted  infantry,  and  helicopters. 

•  The  M1A2  SEPv3  is  expected  to  support  the  full  range  of 
military  operations  by  being  fully  integrated,  expeditionary, 
networked,  decentralized,  adaptable,  and  lethal. 

Major  Contractor 

General  Dynamics  Land  Systems  -  Sterling  Heights,  Michigan 


Activity 

•  DOT&E  approved  the  M1A2  SEPv3  Test  and  Evaluation  *  In  FY15,  the  Army  continued  testing  to  characterize  M1A2 

Master  Plan  and  LFT&E  strategy  on  March  26,  2015.  SEPv3  armor  performance  against  multiple  threat  types 
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under  the  auspices  of  NEA,  a  separate  materiel  development 
verification  and  production  effort.  DOT&E  is  following 
the  NEA  development  and  verification  program  to  leverage 
all  relevant  data  to  support  the  M1A2  SEPv3  survivability 
assessment.  DOT&E  is  working  with  the  Army  to  finalize 
M1A2  SEPv3  armor  performance  evaluation  plans. 

•  In  accordance  with  DOT&E-approved  test  plans,  the  Army 
continued  underbody  IED  T&E  against  M1A2  BH&T  in 
FY15,  to  finalize  design  plans  intended  to  improve  M1A2 
SEPv3  IED  protection. 

•  The  Army  will  conduct  additional  testing  in  F Y 1 6  to  better 
characterize  the  protection  provided  by  the  tank  equipped  with 
the  new  underbody  kit  and  recently  integrated  vulnerability 
reduction  features.  These  features  include  redesigned  crew 
seating,  additional  floor  stiffners,  hardware  to  provide  lower 
limb  protection,  and  changes  in  the  material  and  dimensions  of 
internal  structural  supports. 

•  In  FY16,  the  Army  plans  to  continue  testing  to  characterize 
NEA  and  explosive  reactive  armor  performance,  vulnerabilities 
associated  with  stowed  ammunition  and  underbody  IED 
protection. 


Assessment 

•  DOT&E  continues  to  assess  data  resulting  from  the  Army’s 
ongoing  efforts  to  characterize  the  protection  provided  by  NEA 
against  expected,  operationally-realistic  threats. 

•  The  underbody  IED  testing  against  non-operational  BH&Ts 
with  underbody  protection  kits  has  resulted  in  design  changes 
expected  to  improve  the  M1A2  SEPv3’s  IED  protection. 

•  All  testing  conducted  thus  far  will  inform  the  planning  process 
for  the  final  full-up  system-level  live  fire  testing  of  operational 
M1A2  SEPv3  MBTs  scheduled  in  FY19. 

•  DOT&E  will  leverage  all  relevant  vulnerability  test  data  from 
the  armor  characterization  and  underbody  IED  test  phases  and 
evaluate  all  modeling  and  simulation  tools  available  to  support 
a  FY20  final  assessment  of  the  tank’s  survivability  to  current 
and  expected  threats. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY15  Recommendations.  None. 
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Distributed  Common  Ground  System  -  Army  (DCGS-A) 


Executive  Summary 

•  The  Army  conducted  FOT&E  of  the  Distributed  Common 
Ground  System  -  Army  (DCGS-A)  Increment  1,  Release  2 
at  Fort  Bliss,  Texas,  from  May  2  -  14,  2015,  during  Network 
Integration  Evaluation  15.2. 

•  The  Program  Office  conducted  additional  laboratory  testing 
in  September  2015  to  supplement  the  FOT&E  evaluation  of 
DCGS-A  Release  2. 

•  The  DCGS-A  Increment  1,  Release  2  is  operationally 
effective.  System  availability  and  training  were  adequate, 
but  the  users  rated  the  usability  low-marginal.  The  system 
is  not  survivable  against  cybersecurity  threats  due  to  the 
vulnerability  of  the  Army  network. 

System 

•  The  DCGS-A  provides  an  organic  net-centric  Intelligence, 
Surveillance,  and  Reconnaissance  (ISR)  capability 

from  Battalion  to  Echelons  Above  Corps  by  combining 
16  stove-piped  legacy  systems  of  record  into  one 
comprehensive  network,  including  the  capability  to  process 
Top  Secret/ Sensitive  Compartmented  Information. 

•  To  resolve  shortcomings  discovered  during  the  IOT&E  in 
2012,  the  Army  reconfigured  the  system  as  Release  1  with 
only  the  Secret-level  components.  The  Defense  Acquisition 
Executive  approved  the  full  deployment  of  this  configuration. 

•  The  Army  developed  Release  2  to  improve  the  capabilities  that 
did  not  work  effectively  with  Release  1 .  Release  2  is  intended 
to  provide  enhanced  capabilities  to  include: 

-  Top  Secret/Sensitive  Compartmented  Information 
capability 

-  Workflows  that  aligned  with  how  an  intelligence  section 
would  employ  the  system 

-  Methods  for  transferring  data  within  the  system  and 
between  systems  more  efficiently 

-  Improved  database  structure 

-  Enhanced  fusion  software  for  correlation  of  intelligence 
data 

-  New  materiel  solution  for  transfer  of  information  across 
security  domains 


Intel  Processing  Center  (V)2 
DIV/BCT/BfSB 
Multi-INT  TPED 

DCGS'A  Tactical  Intel 
Ground  Station  (TGS) 
Corps/DIV/BCT/E-MIB 


AEB  -  Area  Exploitation  Battalion 

ASCC  -  Army  Service  Component  Command 

BCT  -  Brigade  Combat  Team 

Bde  -  Brigade 

BfSB  -  Battlefield  Surveillance  Brigade 
DIV  -  Division 

E-MIB  -  Military  Intelligence  Battalion,  Expeditionary 


GEOINT  -  Geospatial  Intelligence 

MFWS  -  Multi-Function  Work  Station 

Ml  -  Military  Intelligence 

P-MFWS  -  Portable  Multi-Function  Work  Station 

TPED  -  Task,  Process,  Exploit  and  Disseminate 

WS  -  Work  Station 


•  In  December  2014,  USD(AT&L)  approved  modification  to 
the  acquisition  strategy  to  end  Increment  1  with  completion  of 
Release  2  deployment.  Requirements  that  were  allocated  to 
Release  3,  to  include  a  cloud-computing  capability  to  support 
worldwide  intelligence  analysis;  database  synchronization;  and 
operations  in  disconnected  or  low-bandwidth  environments, 
will  now  be  allocated  to  Increment  2. 

Mission 

Army  intelligence  analysts  use  DCGS-A  to  perform:  receipt 
and  processing  of  select  ISR  sensor  data,  intelligence 
synchronization,  ISR  planning,  reconnaissance  and  surveillance 
integration,  fusion  of  sensor  information,  and  direction  and 
distribution  of  relevant  threat,  non-aligned,  friendly  and 
environmental  (weather  and  geospatial)  information. 

Major  Contractors 

•  Lockheed  Martin  -  Gaithersburg,  Maryland 

•  ManTech  -  Belcamp,  Maryland 

•  Textron  -  Austin,  Texas 

•  Northrop  Grumman  -  Sacramento,  California 


Activity 

•  On  December  4,  2014,  USD(AT&L)  issued  an  Acquisition 
Decision  Memorandum  (ADM)  that  ended  the  acquisition 
of  Increment  1  with  completion  of  Release  2  deployment. 
Previously,  the  DCGS-A  Increment  1  was  composed  of  three 
releases. 

•  The  Full  Deployment  Decision  ADM,  dated 
December  14,  2014,  required  a  plan  for  a  developmental 


test  with  a  representative  operational  test  network  structure 
using  the  scenarios  and  data  collection/reduction  tools 
expected  to  be  used  for  the  operational  test.  The  Army 
planned  and  conducted  Developmental  Test  2  (DT2)  at 
Fort  Huachuca,  Arizona,  from  September  13  -  27,  2014,  to 
fulfill  the  ADM  requirement. 
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•  The  U.S.  Army  Test  and  Evaluation  Command  (ATEC) 
conducted  an  FOT&E  at  Fort  Bliss,  Texas,  during  the  Army’s 
Network  Integration  Evaluation  15.2  in  May  2015. 

•  ATEC  failed  to  adequately  collect,  reduce,  and  analyze 
FOT&E  test  data  in  accordance  with  the  DOT&E-approved 
test  plan. 

•  The  Program  Office  conducted  a  laboratory  test 
September  14  -  24,  2015  to  provide  data  for  more  rigorous 
evaluation  of  DCGS-A  Release  2  data  synchronization. 

•  DOT&E  will  issue  a  report  on  the  DCGS-A  Release  2  FOT&E 
in  early  FY16. 

Assessment 

•  DT2  data  were  not  sufficient  for  DOT&E  to  evaluate  DCGS-A 
system  performance.  The  Army’s  data  collection,  reduction, 
and  analysis  process  failed  to  provide  adequate  quantitative 
answers  to  the  key  measures  of  performance. 

•  During  the  FOT&E,  the  test  unit  successfully  employed 
DCGS-A  to  locate  and  take  actions  on  all  of  the  injected 
terrorist  activities.  The  unit  also  accurately  tracked  the  enemy 
troops  and  equipment  movements,  but  did  not  always  attribute 
the  troops  and  equipment  to  the  correct  enemy  unit.  The 
Army  injected  supporting  intelligence  data  for  10  intelligence 
vignettes  into  the  test  database,  which  also  contained  terabytes 
of  other  operationally  representative  intelligence  data.  During 
the  test,  the  test  unit  successfully  discovered  and  exploited 
the  supporting  data  for  all  1 0  vignettes  and  drew  appropriate 
conclusions  from  the  data  within  hours. 

•  FOT&E  data  collected  by  ATEC  were  not  adequate  to 
quantitatively  evaluate  fusion,  targeting,  and  database 
synchronization.  The  program  manager  conducted  a  test  of 
data  synchronization  in  an  operationally  realistic  laboratory 
facility  in  September  2015  and  provided  data  to  supplement 
the  evaluation  of  data  synchronization.  Positive  results  of  the 
vignettes  indicate  the  fusion  capability  was  adequate  to  support 
the  mission  during  FOT&E. 

•  The  data  from  the  excursion  showed  that  DCGS-A  data 
synchronization  can  work  effectively  if  the  most  efficient 
method  is  used.  Users  can  choose  from  four  different  ways 
of  synchronizing  the  data.  During  the  FOT&E,  the  unit  chose 
to  use  the  ad-hoc  Datamover  method  because  they  perceived 
this  to  be  the  most  flexible  method.  The  lab  test  showed  using 
this  method  with  a  large  number  of  entities  (about  900  entities; 
the  number  emulates  the  database  used  during  the  FOT&E) 
could  take  about  2  hours,  whereas  moving  the  same  data 


with  a  scheduled  Datamover  can  be  completed  in  less  than 
20  minutes.  The  Army  plans  to  modify  training  to  use  the 
scheduled  Datamover  to  synchronize  large  numbers  of  entities. 

•  DCGS-A  availability  was  0.99,  satisfying  the  requirement  of 
0.90.  Reliability,  in  terms  of  Mean  Time  Between  Failure, 
ranged  from  16  to  360  hours  depending  on  the  location  and 
functions.  Reliability  and  maintainability  were  sufficient  to 
conduct  the  mission,  but  improvement  in  reliability  would 
improve  DCGS-A  suitability. 

•  The  system  usability  scale  indicated  system  usability  to  be 
low-marginal. 

•  The  system  was  not  survivable  against  cybersecurity  threats 
because  of  the  vulnerabilities  in  the  Army’s  tactical  network. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  did  not 
successfully  implement  the  FY14  recommendation  to 
incorporate  lessons  learned  from  DT2  to  conduct  the  FOT&E; 
the  FOT&E  data  collection,  reduction,  and  analysis  had 
significant  systematic  shortfalls  similar  to  those  experienced 
during  DT2. 

•  FY 15  Recommendations.  The  Army  should  take  the  following 
actions: 

1.  Institutionalize  the  training  provided  to  the  FOT&E  test 
unit,  so  that  all  DCGS-A  equipped  units  receive  intensive, 
scenario-driven,  collective  training. 

2.  Maintain  DCGS-A  unit  readiness  via  continuous  use  of 
DCGS-A  in  garrison. 

3.  Improve  the  cybersecurity  posture  in  all  Army  tactical 
networks. 

4.  ATEC  should  resolve  systematic  shortfalls  with  data 
collection,  reduction,  and  analysis  during  testing. 

-  Demonstrate  the  end-to-end  process  of  collecting, 
reducing,  and  analyzing  the  data  before  an  operational 
test. 

-  Conduct  a  developmental  test  with  operationally 
representative  networks  and  the  operational  test 
instrumentation  before  an  operational  test  of  complex 
networked  systems. 

-  Attribute  all  performance  anomalies  to  system 
performance,  test  process,  or  data  collection  and 
reduction  before  the  test  ends. 

-  Analyze  data  sufficiently  to  identify  and  resolve 
anomalies  and  inconsistencies  during  the  test. 
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Global  Combat  Support  System  -  Army  (GCSS-Army) 


Provides 
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Executive  Summary 

•  From  March  30  to  April  10,  2015,  DOT&E 
monitored  a  Lead  Site  Verification 
Test  (LSVT)  of  Wave  2  capability 
enhancements  at  two  active  Army  units 
(2nd  Heavy  Brigade  Combat  Team,  1st 
Armored  Division,  and  11th  Armored 
Cavalry  Regiment);  one  Army  Reserve 
unit  (94th  Training  Division);  and  one 
Army  National  Guard  unit  (60th  Troop 
Command). 

•  From  January  2015  through  June  2015, 

DOT&E  analyzed  system  performance  data 
to  assess  system  scalability  associated  with 
Material  Requirements  Planning  (MRP) 
batch  jobs  as  the  Army  expanded  the  user 
base  through  continued  fielding  of  the 
Global  Combat  Support  System  -  Army 
(GCSS-Army)  to  additional  units. 

•  GCSS-Army  Wave  2  is  operationally  effective.  The  system 
successfully  surpassed  the  90  percent  threshold  for  all  Wave 
2  critical  mission  functions  attempted  by  users.  Reports 
generated  at  all  levels  provided  leaders  with  essential 
decision-making  information  to  support  force  maintenance  and 
sustainment. 

•  GCSS-Army  is  operationally  suitable,  with  usability  and 
resolution  of  help  desk  tickets  needing  some  improvement. 
GCSS-Army  exceeded  the  requirements  for  system  reliability 
and  availability. 

•  The  system  is  survivable  against  an  intermediate-level  outsider 
threat,  but  is  vulnerable  to  an  intermediate-level  insider  cyber 
threat.  Survivability  against  an  advanced  persistent  outsider 
cyber  threat  was  not  tested. 

•  The  GCSS-Army  program  continues  to  make  progress  in 
support  of  the  legislative  mandate  to  be  financially  auditable 
by  2017. 
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•  The  hardware  component  of  GCSS-Army  is  located  on 
production  servers  in  Redstone,  Alabama,  and  Continuity  of 
Operations  servers  in  Radford,  Virginia. 

•  The  GCSS-Army  program  includes  the  Army  Enterprise 
Systems  Integration  Program  that  provides  the  enterprise 
hub  services,  centralized  master  data  management,  and 
cross-functional  business  intelligence  and  analytics  for  Army 
Enterprise  Resource  Planning  solutions,  including  the  General 
Fund  Enterprise  Business  System  and  Logistics  Modernization 
Program. 

•  The  Army  is  fielding  GCSS-Army  in  two  waves: 

-  Wave  1  contains  the  retail  supply  and  associated  financial 
functions  and  will  be  completed  in  2QFY16. 

-  Wave  2  contains  the  remaining  functions  and  will  be 
fielded  in  FY16-17. 

•  GCSS-Army  executes  financial  actions  and  is  therefore  subject 
to  the  2010  National  Defense  Authorization  Act  requirement 
to  be  auditable  by  2017. 


System 

•  GCSS-Army  is  an  information  technology  system  based 
on  commercial  off-the-shelf  and  government  off-the-shelf 
software  and  hardware. 

•  GCSS-Army  uses  an  adaptation  of  a  commercially-available 
Enterprise  Resource  Planning  system  to  integrate  internal 
and  external  management  information  across  an  organization, 
including  finance/accounting,  manufacturing,  sales  and 
service,  and  customer  relationship  management.  GCSS-Army 
centralizes  and  standardizes  these  activities  and  provides 
automation  to  assist  users  with  common  tasks,  such  as 
reporting. 


Mission 

Army  logisticians  use  GCSS-Army  to  view,  access,  and  exchange 
consolidated  operational  logistics  data  to  conduct  maintenance, 
material  management,  property  accountability,  financial 
management,  and  logistics  planning. 

Major  Contractors 

•  Northrop  Grumman  Space  and  Mission  Systems 
Division  -  Bon  Air,  Virginia 

•  LMI  Consulting  -  McLean,  Virginia 

•  InSAP  Services,  Inc.  -  Marlton,  New  Jersey 
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Activity 

•  The  Army  Research  Laboratory  Snrvivability/Lethality 
Analysis  Directorate  conducted  cybersecurity  vulnerability 
and  penetration  assessment  testing  of  GCSS-Army  in  January 
2015  in  accordance  with  a  DOT&E-approved  test  plan.  The 
purpose  of  this  testing  was  to  identify  vulnerabilities  and  allow 
time  to  fix  them  prior  to  a  full-scope  Adversarial  Assessment 
and  Cyber  Economic  Vulnerability  Assessment. 

•  From  January  2015  through  June  2015,  DOT&E  analyzed 
system  performance  data  to  assess  system  scalability 
associated  with  MRP  batch  jobs  as  the  Army  expanded  the 
user  base  through  continued  fielding  of  GCSS-Army  to 
additional  units. 

•  A  Red  Team  from  the  Army’s  Threat  Systems  Management 
Office  (TSMO)  conducted  a  full-scope  cybersecurity 
Adversarial  Assessment  and  cyber  economic  vulnerability 
testing  of  GCSS-Army  in  February  2015.  TSMO  received 
support  from  a  commercial  financial  auditing  team  for  the 
cyber  economic  vulnerability  testing. 

•  From  February  23  through  March  13,  2015,  DOT&E  observed 
independent  government  testing  of  deployment/redeployment 
functionality  to  and  from  both  mature  and  immature  theaters 
of  operation.  This  was  an  integrated  developmental  test/ 
operational  test  conducted  at  Fort  Lee,  Virginia,  with  Active 
Duty  and  Army  National  Guard  Soldiers  using  the  Program 
Office’s  developmental  test  client. 

•  From  March  23  through  April  10,  2015,  DOT&E  monitored 
an  LSVT  of  Wave  2  capability  enhancements  at  two  active 
Army  units  (2nd  Heavy  Brigade  Combat  Team,  1st  Armored 
Division  at  Fort  Bliss,  Texas,  and  11th  Armored  Cavalry 
Regiment  at  Fort  Irwin,  California);  one  Army  Reserve  unit 
(94th  Training  Division  at  Fort  Bragg,  North  Carolina); 
and  one  Army  National  Guard  unit  (60th  Troop  Command, 
Raleigh,  North  Carolina).  This  was  an  independent 
operational  test  involving  typical  users  in  an  operationally 
realistic  environment  to  assess  specific  risk  factors  for 
operational  effectiveness,  operational  suitability,  and 
survivability. 

•  From  May  26  through  June  9,  2015,  the  Army  Test  and 
Evaluation  Command  (ATEC)  witnessed  independent 
government  testing  of  disconnected  automated  identification 
technology  capabilities  at  the  Program  Management  Office’s 
facility  in  Petersburg,  Virginia.  This  technology  allows 
GCSS-Army  users  to  conduct  limited  supply  operations 
without  an  active  communications  network. 

•  From  August  17  through  September  3,  2015,  ATEC  reviewed 
the  end-to-end  deployment/redeployment  standard  operating 
procedure  documentation  at  the  Program  Office’s  facility 

in  Petersburg,  Virginia.  This  documentation  will  be  used 
by  Army  commanders  to  orchestrate  deployment  processes 
using  GCSS-Army.  Additionally,  ATEC  observed  follow-on 
independent  government  testing  of  disconnected  automated 
identification  technology  capabilities. 


•  A  full  transfer  of  operations  to  and  from  the  continuity  of 
operations  location  was  not  tested. 

•  DOT&E  submitted  an  FOT&E  report  in  November  2015  on 
the  LSVT. 

Assessment 

•  GCSS-Army  Wave  2  is  operationally  effective. 

-  The  system  successfully  surpassed  the  90  percent  threshold 
for  all  Wave  2  critical  mission  functions  attempted  by 
users. 

-  Reports  generated  at  all  levels  provided  leaders  with 
essential  decision-making  information  to  support  force 
maintenance  and  sustainment.  However,  users  noted  that 
some  maintenance,  finance,  and  logistics  management 
reports  took  longer  than  expected  to  run  or  they  timed  out 
before  completion,  causing  users  to  spend  more  time  with 
multiple  transaction  attempts. 

-  Server  capacity  can  support  continued  Wave  1  fielding  and 
the  continuation  of  upgrades  from  Wave  1  to  Wave  2. 

-  During  the  LSVT,  the  new  interface  between  GCSS-Army 
and  the  legacy  Standard  Army  Management  Information 
System  systems  for  property  book,  maintenance,  and  unit 
supply  worked  properly. 

•  GCSS-Army  is  operationally  suitable,  with  usability  and 
resolution  of  help  desk  tickets  needing  some  improvement. 
GCSS-Army  exceeded  the  requirements  for  system  reliability 
and  availability. 

•  GCSS-Army  was  shown  to  be  survivable  against  an 
intermediate-level  outsider  cyber  threat,  but  was  vulnerable 
to  an  intermediate-level  insider  cyber  threat.  Survivability 
against  an  advanced  persistent  outsider  cyber  threat  using 
specialized  tools  or  exploits  was  not  tested.  GCSS-Army  has 
improved  its  cybersecurity  capabilities  since  earlier  testing. 

•  GCSS-Army  demonstrated  the  ability  to  detect  and  react 
to  cyber  threats  in  support  of  the  operational  mission,  data, 
and  users.  A  Cyber  Economic  Vulnerability  Assessment, 
performed  as  a  table-top  assessment  based  on  vulnerabilities 
discovered  and  exploited  during  the  Adversarial  Assessment, 
did  not  reveal  any  additional  significant  risks.  While  analysis 
shows  that  the  MRP  reports  are  running  slightly  longer,  it  is 
not  indicative  of  a  capacity  shortfall. 

•  During  the  migration  to  GCSS-Army  Wave  2,  the  number 
of  units  running  reports  and  the  number  of  reports  they  are 
running  was  expected  to  increase.  The  weekly  MRP  report 
data  provides  the  Program  Office  the  ability  to  predict  any 
future  need  to  upgrade  its  servers  to  handle  the  increasing 
report  processing  workload. 

•  The  2010  National  Defense  Authorization  Act  requires 
financial  audibility  by  2017.  GCSS-Army  continues  to 
work  to  achieve  certification  in  accordance  with  the  Federal 
Financial  Management  Improvement  Act  through  various 
audits. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  GCSS-Army 
program  manager  has  addressed  all  previous  recommendations 
and  continues  to  make  progress  in  support  of  meeting  the 
legislative  mandate  to  be  financially  auditable  by  2017. 


•  FY15  Recommendation. 

1.  The  GCSS-Army  Program  Office  should  conduct  a 
full  transfer  of  operations  to  and  from  the  continuity  of 
operations  location. 


GCSS-Army 
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Guided  Multiple  Launch  Rocket  System  -  Alternate 
Warhead  (GMLRS-AW)  M30E1 


Executive  Summary 

•  The  M30E1  Guided  Multiple  Launch  Rocket 

System  -  Alternate  Warhead  (GMLRS-AW)  surface-to-surface 
rocket  meets  the  DOD  unexploded  ordnance  policy 
requirements. 

•  The  Army  conducted  the  GMLRS-AW  IOT&E  from 
October  through  November  2014  at  White  Sands  Missile 
Range.  As  part  of  the  IOT&E,  the  Army  conducted  an 
Adversarial  Assessment  from  October  15  -  17,  2014. 

•  On  March  26,  2015  DOT&E  submitted  a  classified  combined 
IOT&E/LFT&E  report  detailing  the  results  of  testing.  The 
Army  Acquisition  Executive  approved  full-rate  production  on 
April  8,2015. 

•  Based  on  IOT&E  results,  DOT&E  recommended  the  Army 
update  GMLRS-AW  tactics,  techniques,  and  procedures 
(TTP).  The  Army  revised  the  GMLRS-AW  targeting 
procedures. 

•  Using  the  revised  targeting  TTP,  the  Army  conducted  a 
two-mission  follow-on  test  at  White  Sands  Missile  Range 
from  May  19  -  22,  2015,  to  address  recommendations 
obtained  from  the  IOT&E. 

•  In  November  2015,  DOT&E  submitted  an  FOT&E  classified 
report  detailing  the  results  of  follow-on  testing  and  assessed 
the  following: 

-  GMLRS-AW  contains  no  submunitions  to  cause 
unintended  harm  to  civilians  and  infrastructure  associated 
with  unexploded  ordnance  from  cluster  munitions. 

-  GMLRS-AW  is  operationally  effective  with  the  Army’s 
updated  targeting  TTP. 

-  GMLRS-AW  is  accurate.  The  GMLRS-AW  rocket  is  well 
within  the  required  specification. 

-  GMLRS-AW  is  operationally  suitable.  Including  the 
follow-on  test,  99  rockets  were  fired  during  developmental 
and  operational  testing.  All  99  rockets  were  reliable. 

-  GMLRS-AW  is  survivable. 

System 

•  The  200-pound  GMLRS-AW  high-explosive  rocket  contains 
approximately  182,000  pre-formed  tungsten  fragments. 
GMLRS-AW  M30E 1  surface-to-surface  rocket  meets  the 
2008  DOD  Policy  on  Cluster  Munitions  and  Unintended  Harm 
to  Civilians. 


•  The  GMLRS-AW  rocket  uses  Inertial  Measurement  Unit  and 
GPS  guidance  augmentation  to  engage  area  targets  out  to  a 
range  of  70  kilometers. 

•  GMLRS-AW  uses  the  same  rocket  motor,  guidance  system, 
and  control  system  as  the  existing  M3 1A1  GMLRS  Unitary 
rocket. 

•  The  GMLRS-AW  rockets  can  be  fired  from  the  tracked 
M270A1  Multiple  Launch  Rocket  System  and  the  wheeled 
High  Mobility  Artillery  Rocket  System. 

•  The  procurement  objective  is  18,072  GMLRS-AW  rockets. 
The  Army  entered  full-rate  production  on  April  8,  2015. 

Mission 

Commanders  will  use  GMLRS-AW  rockets  to  engage  area-  or 

imprecisely-located  targets  without  the  hazard  of  unexploded 

submunitions. 

Major  Contractor 

Lockheed  Martin  Missiles  and  Fire  Control  -  Dallas,  Texas 


Activity 

•  The  Army  conducted  the  GMLRS-AW  IOT&E  from  Range.  As  part  of  the  IOT&E,  the  Army  conducted  an 

October  through  November  2014,  at  White  Sands  Missile  Adversarial  Assessment  from  October  15  -  17,  2014. 
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•  DOT&E  submitted  a  classified  combined  IOT&E/LFT&E 
report  on  March  26,  2015,  detailing  the  results  of  testing.  The 
Army  Acquisition  Executive  approved  the  full-rate  production 
on  April  8,  2015. 

•  From  May  19  -  22,  2015,  the  Army  conducted  a  two-mission 
follow-on  test  to  demonstrate  the  newly-developed  TTP. 

The  Army  fired  the  new  TTP  solutions  at  the  follow-on  test. 
DOT&E  submitted  a  classified  Operational  Assessment  report 
on  November  3,  2015  detaining  the  results  of  the  testing. 

•  The  Army  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  test  plans. 

Assessment 

•  GMLRS-AW  munition  does  not  contain  submunitions  to  cause 
unintended  harm  to  civilians  and  infrastructure  associated  with 
unexploded  ordnance  from  cluster  munitions  and  it  meets  the 
dud  rate  requirement. 

•  GMLRS-AW  is  operationally  effective  with  the  Army’s 
updated  targeting  TTP. 

-  In  the  IOT&E,  GMLRS-AW  met  the  Army’s  effectiveness 
requirements  for  10  of  12  missions.  A  unit  equipped 
with  GMLRS-AW  was  not  effective  for  certain  targets. 
Details  can  be  found  in  DOT&E’s  classified  report  on 
GMLRS-AW  dated  March  26,  2015. 

-  Using  IOT&E  results,  the  Army  developed  new 
GMLRS-AW  targeting  TTPs. 

•  Targets  executed  in  the  follow-on  test  had  the  same 
targeting  errors  and  countermeasures  as  the  original  IOT&E 


missions.  Both  missions  met  the  Army’s  effectiveness 
requirements.  Details  of  the  follow-on  test  can  be  found  in 
DOT&E’s  classified  Operational  Assessment  report  dated 
November  3,  2015. 

•  The  GMLRS-AW  rocket  is  accurate.  During  the  IOT&E 
and  follow-on  test  the  GMRLS-AW  rocket  is  well  within  the 
required  specification. 

•  GMLRS-AW  is  suitable.  Including  the  follow-on  test, 

99  rockets  were  fired  during  developmental  and  operational 
testing.  All  99  rockets  were  reliable. 

•  GMLRS-AW  is  survivable.  No  cybersecurity  vulnerabilities 
were  found  with  the  rocket  or  launcher.  Some  vulnerabilities 
were  discovered  with  the  missile  test  device  used  at  depot  to 
test  rocket  hardware.  The  Program  Office  is  addressing  the 
new  cybersecurity  issues. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed  all 
previous  recommendations. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Model  the  ability  of  a  committed  force  to  sustain 
GMLRS-AW  munitions  in  full  spectrum  operations  given 
the  increase  in  rockets  to  manage  the  counter  fire  campaign. 

2.  Model  the  effectiveness  of  GMLRS-AW  munitions  against 
targets  with  different  types  of  countermeasures. 
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HELLFIRE  Romeo 


Executive  Summary 

•  The  HELLFIRE  missile  (AGM-1 14)  is  a  family  of 
air-to-surface,  guided  munitions  consisting  of  missile  body 
with  different  warhead  types. 

•  The  Air  Force  developed  a  new  warhead,  the  HELLFIRE 
Romeo  missile  variant,  to  provide  increased  lethality  against  a 
variety  of  non-traditional  targets. 

•  The  Air  Force  will  operate  the  HELLFIRE  from  MQ-1 
Predator  and  MQ-9  Reaper  unmanned  aerial  vehicles  (UAVs). 
DOT&E  assessed  the  HELLFIRE  Romeo  missile  variant  as 
lethal. 

•  The  Air  Force  authorized  fielding  in  December  2014, 
following  an  interim  report  submitted  by  DOT&E  in 
November  2014.  DOT&E  submitted  the  final  report  on  the 
HELLFIRE  Romeo  missile  variant  in  August  2015,  after  the 
Air  Force  completed  the  HELLFIRE  Romeo  lethality  testing 
against  maritime  targets. 

System 

•  The  AGM-1 14  HELLFIRE  is  a  family  of  laser  guided  missiles 
for  use  against  fixed  and  moving  targets  by  both  rotary  and 
fixed-wing  aircraft  (including  UAVs). 

•  The  HELLFIRE  Romeo  missile  variant: 

-  Is  an  air-to-surface  missile  intended  to  be  launched  from 
Air  Force  UAV  platforms.  It  uses  a  new  warhead  and  a 
semi-active  laser  seeker  to  home-in  on  its  target. 

-  Will  replace  the  HELLFIRE  K2A  fragmenting  warhead 
variant  and  supplement  the  existing  HELLFIRE  R2 
anti-armor  variant  currently  fielded  by  the  Air  Force  for 
air-to-surface  engagements. 

-  Is  designed  to  provide  improved  lethality  against 
combatants  within  building  structures  while  maintaining 
lethality  against  non-armored  targets. 


-  Is  compatible  with  other  HELLFIRE  missiles  fired  from 
other  Air  Force  UAVs.  Like  other  HELLFIRE  variants,  the 
HELLFIRE  Romeo  missile  includes  variable  time  delay 
fuzing  options. 

Mission 

Commanders  will  employ  the  HELLFIRE  Romeo  missile  variant 
to  engage  enemy  combatants  located  within  complex  building 
and  bunker  structures,  in  non-armored  vehicles,  in  small  boats, 
and  in  the  open  from  UAVs. 

Major  Contractor 

Lockheed  Martin  Corporation,  Missiles  and  Fire  Control 
Division  -  Grand  Prairie,  Texas 

(The  missiles  are  manufactured  in  Ocala,  Florida,  and  Troy, 
Alabama.) 


Activity 

•  The  Air  Force  successfully  completed  live  fire  testing  of  the 
HELLFIRE  Romeo  missile  in  February  2015.  Testing  was 
conducted  in  accordance  with  the  DOT&E-approved  live  fire 
strategy  and  test  plans. 

•  The  HELLFIRE  Romeo  missile  LFT&E  program  included 
arena  tests,  developmental  dynamic  tests  against  masonry 
targets,  developmental  flight  tests  against  building  and  bunker 
targets,  missile  flight  tests  against  mannequins,  trucks,  light 
armor,  buildings,  and  bunkers,  a  rocket-on-a-rope  test  against 
a  boat  target,  and  a  range  of  supporting  lethality  assessments 
using  modeling  and  simulation. 


•  The  Air  Force  approved  fielding  of  the  HELLFIRE  Romeo 
missile  variant  in  late  2014  following  DOT&E’s  submission  of 
an  interim  classified  lethality  report. 

•  DOT&E  submitted  a  final  classified  lethality  report  for 
the  HELLFIRE  Romeo  missile  in  August  2015  after  the 
completion  of  the  boat  target  testing. 

Assessment 

•  The  HELLFIRE  Romeo  missile  demonstrated  adequate 
lethality  across  a  spectrum  of  expected  targets,  inc  luding  small 
boats,  light  armor,  technical  vehicles  (trucks),  and  personnel 


HELLFIRE  Romeo 


115 


F  Y  1  5  ARMY  PROGRAMS 


both  in  the  open  and  behind/under  a  variety  of  masonry 
structures. 

•  The  classified  lethality  report  identified  engagement 
circumstances  and  target  conditions  for  which  HELLFIRE 
Romeo  lethality  against  specific  targets  either  is  not  known,  or 
which  affect  lethality  against  a  particular  target. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 


•  FY15  Recommendations.  The  Air  Force  or  the  FIELLFIRE 

Romeo  program  should: 

1.  Address  the  three  recommendations  in  the  classified  report 
to  further  quantify  lethality  estimates  against  specific  targets 
in  specific  conditions  and  engagement  circumstances. 

2.  Provide  the  classified  test  results  to  the  Joint  Technical 
Coordinating  Committee  for  Munitions  Effectiveness 
(JTCG/ME)  for  incorporation  into  JTCG/ME  products  as 
indicated  in  the  final  classified  DOT&E  report. 
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Integrated  Personnel  and  Pay  System  -  Army  (IPPS-A) 
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Executive  Summary 

•  Integrated  Personnel  and  Pay  System  -  Army 
(IPPS-A)  is  a  two  increment  program 
that  streamlines  Army  Human  Resources 
processes  and  enhances  the  efficiency 
and  accuracy  of  Army  personnel  and  pay 
procedures  to  support  Soldiers  and  their 
families.  Through  a  three  phased  delivery 
approach.  Increment  1  of  the  IPPS-A  program 
provides  the  foundational  data  for  a  single, 
integrated  military  personnel  and  pay 
system  for  all  three  Army  components:  the 
active-duty  Army,  the  Army  Reserve  National 
Guard,  and  the  Army  Reserve. 

•  The  Army  Test  and  Evaluation  Command 
(ATEC)  conducted  a  two-part  FOT&E  event 
from  March  2014  through  January  2015  in 
accordance  with  an  ATEC-approved  test  plan. 

•  IPPS-A,  as  it  exists  in  Increment  I,  is  effective 
and  suitable.  IPPS-A  is  survivable  against 

an  outsider  cyber  threat.  The  capabilities 
available  in  this  increment  are  limited;  the 
program  should  continue  to  improve  IPPS-A 
in  order  to  deliver  the  full  set  of  necessary 
capabilities. 

•  The  IPPS-A  Increment  I  system  demonstrated  the  capability  to 
produce  its  primary  product,  a  Soldier’s  Record  Brief  (SRB), 
which  is  a  single,  integrated  compilation  of  personnel  and  pay 
data  collected  from  various,  external  authoritative  sources. 

System 

•  IPPS-A  is  a  two  increment  program  that  streamlines  Army 
Human  Resources  processes  and  enhances  the  efficiency  and 
accuracy  of  Army  personnel  and  pay  procedures  to  support 
Soldiers  and  their  families.  Increment  I  interfaces  with  legacy 
applications  to  create  a  trusted,  foundational  database.  All 
authoritative  data  remain  in  the  legacy  systems  for  Increment 
I.  Increment  II  will  become  the  authoritative  data  source 

as  the  necessary  functionality  of  the  legacy  systems  to  be 
subsumed  is  incorporated. 

•  IPPS-A  is  a  web-based  tool,  available  24  hours  a  day, 
accessible  to  Soldiers,  Human  Resources  professionals, 
Combatant  Commanders,  personnel  and  pay  managers,  and 
other  authorized  users  throughout  the  Army.  The  Army 
intends  for  IPPS-A  to  improve  the  delivery  of  military 
personnel  and  pay  services  and  provide  internal  controls  and 
audit  procedures  to  prevent  erroneous  payments  and  loss  of 
funds. 
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•  IPPS-A  interfaces  with  15  other  Army  and  DOD  systems 
to  acquire  personnel  and  pay  data,  which  it  integrates 
into  a  single  record  for  each  Soldier.  These  systems 
include  the  Defense  Enrollment  Eligibility  Reporting 
System  -  Personnel  Data  Repository,  Electronic  Military 
Personnel  Office,  Standard  Installation  and  Division  Personnel 
Reporting  System  -  Guard,  and  Total  Army  Personnel  Data 
Base  -  Reserve.  IPPS-A’s  SRB  acts  as  a  trusted,  but  non 
authoritative,  display  of  data  contained  in  the  various  external 
systems;  any  changes  required  to  the  data  must  be  made 
within  the  existing  15  Army  and  DOD  personnel  systems  and 
cannot  be  accomplished  within  IPPS-A.  The  SRB  displays  a 
Soldier’s  military  career  personal  information,  qualification 
skills,  training,  assignment  history,  and  various  other  Soldier 
attributes. 

Mission 

Commanders  will  employ  IPPS-A  as  a  comprehensive  system 
for  accountability  and  information  to  support  command 
decisions  regardless  of  component  or  geographic  location.  Army 
components  will  use  IPPS-A  to  manage  their  members  across 
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the  full  operational  spectrum  during  peacetime,  war,  through 
mobilization  and  demobilization,  capturing  timely  and  accurate 
data  throughout.  Soldiers  will  use  IPPS-A  as  a  single,  integrated 
personnel  and  pay  system  that  will  provide  personnel  and  pay 
management  functionality  for  all  Army  components. 

Major  Contractor 

Increment  I:  EDC  Consulting  LLC  -  McLean,  Virginia 

Activity 

•  ATEC  conducted  a  risk  assessment  of  1PPS-A  Increment  I 
on  June  19,  2014,  in  accordance  with  DOT&E  Information 
and  Business  Systems  Policy.  Due  to  the  low  risk  of  the 
capabilities  being  delivered  in  Increment  I,  the  risk  assessment 
allowed  for  the  delegation  of  test  plan  approval  to  ATEC. 

•  ATEC  conducted  an  FOT&E  event  from  March  2014  through 
January  2015.  The  FOT&E  was  conducted  in  two  phases. 

-  Phase  1  of  the  FOT&E  utilized  email  surveys  of  a  sample 
of  the  worldwide  Army  population  including  Active,  Army 
National  Guard,  and  Army  Reserve  personnel. 

-  Phase  2  included  specific  test  actions  at  Rock  Island 
Arsenal,  Illinois,  and  Camp  Shelby,  Missouri,  in 
January  2015. 

•  In  November  2014,  ATEC  conducted  a  cybersecurity 
assessment  on  IPPS-A  Increment  I;  however,  the  test  had 
several  limitations.  In  April  2015,  ATEC  conducted  a  second 
cybersecurity  assessment,  which  had  fewer  limitations.  During 
these  cybersecurity  assessments,  the  Army’s  Threat  Systems 
Management  Office  (TSMO)  conducted  operations  remotely 
from  Redstone  Arsenal  in  Huntsville,  Alabama,  and  onsite  at 
the  Human  Resources  Command  in  Fort  Knox,  Kentucky,  and 
Acquisition,  Logistics,  and  Technology  Enterprise  System 
and  Services  in  Radford,  Virginia.  Testing  was  conducted  in 
accordance  with  the  DOT&E  Information  Assurance  policy. 

•  DOT&E  submitted  a  classified  FOT&E  report  in  April  2015 
and  a  classified  cybersecurity  report  in  October  2015  detailing 
the  results  of  testing. 

Assessment 

•  IPPS-A  Increment  I,  as  delivered,  provides  an  SRB  that  is 
viewable  through  a  web  interface  and  can  be  printed  out. 
Increment  I  does  not  provide  the  capability  to  add  or  edit 
personnel  data.  The  ability  to  edit  personnel  and  pay  data  will 
be  phased  in  during  the  four  releases  in  Increment  II. 

•  The  results  of  the  online  survey  indicate  the  system  was  easy 
to  use  and  the  resources  necessary  to  obtain  and  interpret  the 
data  on  the  SRB  were  adequate.  The  results  also  indicate 
that  the  training  received  and  online  resources  available  were 
sufficient  for  most  Soldiers.  Very  few  of  the  Soldiers  used  the 
help  desk,  and  the  associated  survey  results  did  not  provide  a 
significant  response  as  to  whether  they  were  satisfied  with  the 
help  desk  support. 

•  The  SRB  can  be  categorized  into  1 1  sections  and  the  Header 
and  Footer  sections.  Participants  found  data  errors  in  all 

11  sections  and  the  Header  and  Footer  sections  of  the  SRB. 
Sections  where  more  than  50  percent  of  the  participants  had 


data  errors  include  Personal/family  data,  Civilian  Education, 
and  Military  Education.  The  FOT&E  found  that  the  legacy 
sources  contributing  to  most  errors  are:  Army  Training 
Requirements  and  Resources  System,  Reserve  Component 
Manpower  System,  and  Standard  Installation  and  Division 
Personnel  Reporting  System. 

•  During  the  FOT&E,  as  previously  observed,  data  correctness 
continues  to  be  a  significant  problem.  Data  displayed  in  the 
SRB,  which  is  pulled  from  legacy  Human  Resources  systems, 
have  many  problems,  including  missing  and/or  incorrect  data. 
In  addition,  93  percent  of  the  Soldiers  surveyed  found  errors 
in  their  records,  as  compared  to  the  results  from  IOT&E  in 
February  2014,  where  95.2  percent  of  the  Soldiers  (181  of 
190)  found  errors.  This  comparison  shows  the  data  correctness 
problem  still  exists  and,  as  the  sample  size  of  users  surveyed 
in  the  FOT&E  is  much  larger  (about  1  percent  of  the  total 
Army  military  population)  than  in  the  IOT&E,  the  results  are 
indicative  of  a  widespread  problem  across  the  Active  Duty 
Army,  Army  Reserve  National  Guard,  and  Army  Reserve. 

•  The  IPPS-A  program  manager  does  not  have  authority  or 
mechanisms  to  correct  data  within  legacy  Human  Resource 
systems.  The  IPPS-A  Program  Management  Office  and  the 
Army  G-l  are  conducting  a  data  correctness  campaign  that 
administers  online  surveys  to  facilitate  reporting  and  resolution 
of  data  correctness  issues.  FOT&E  results  demonstrate  that 
the  Army  is  continuing  to  correct  the  data  once  a  Soldier 
identifies  an  error  and  proactively  initiates  an  action  to  get 

it  corrected.  However,  the  process  is  lengthy  and  difficult 
to  fully  accomplish  due  to  legacy  system  limitations  and 
documentation  requirements. 

•  ATEC  conducted  a  cybersecurity  assessment  of  the  fully 
deployed  database  in  November  2014  using  the  Army’s 
TSMO  to  portray  the  cyber  threat.  TSMO  conducted 
operations  remotely  from  Redstone  Arsenal  in  Huntsville, 
Alabama,  and  onsite  at  the  Human  Resources  Command 
in  Fort  Knox,  Kentucky,  and  Acquisition  Logistics  and 
Technology  Enterprise  System  and  Services  in  Radford, 
Virginia. 

-  The  cybersecurity  threat  was  not  fully  realistic  due  to 
limitations  on  time  and  rules  of  engagement  at  the  Human 
Resources  Command,  Fort  Knox,  Kentucky.  ATEC 
successfully  completed  a  verification  of  fixes  event 
addressing  most  of  the  findings  on  January  16,  2015.  A 
follow-up  cybersecurity  assessment  was  conducted  in 
April  2015. 
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-  The  follow-up  cybersecurity  assessment  was  found  to 
be  threat-representative.  Testing  uncovered  areas  that 
need  survivability  improvement.  This  will  require  the 
data  centers  to  work  with  cyber  defenders  to  improve 
detection  capabilities,  ensuring  1PPS-A  and  interfacing 
systems  are  following  Personally  Identifiable  Information 
(PII)  encryption  policies  and  performing  periodic 
Cooperative  Vulnerability  and  Penetration  Assessments 
and  cybersecurity  assessments  at  the  data  centers  hosting 
IPPS-A. 

•  The  details  of  the  cybersecurity  test  findings  can  be  found  in 
DOT&E’s  classified  cybersecurity  report  dated  October  2015. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  is  making 
satisfactory  progress  in  the  previous  FY14  recommendations. 


•  FY 15  Recommendations.  The  Army  should: 

1.  Continue  to  monitor  data  correctness  with  the  Army  G-l  to 
ensure  IPPS-A  will  have  accurate  data  to  facilitate  accurate 
transactions  in  Increment  II. 

2.  Conduct  a  fully  threat-representative  cybersecurity 
assessment  for  IPPS-A  Increment  I. 

3.  Verify  vulnerabilities  identified  in  the  threat-representative 
cybersecurity  assessment  conducted  in  April  2015  have 
been  mitigated  in  order  to  maintain  the  performance  and 
integrity  of  the  currently  operating  system. 

■  Work  with  cyber  defenders  to  improve  cyber  detection 
capabilities. 

■  Ensure  all  PII  encryption  policies  are  followed. 

■  Perform  periodic  Cooperative  Vulnerability  and 
Penetration  Assessments  and  cybersecurity  assessments 
at  the  data  centers  hosting  IPPS-A. 
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Javelin  Close  Combat  Missile  System  -  Medium 


Executive  Summary 

•  In  F  Y 1 5,  the  Army  continued  development  and  test  of  the 
Javelin  Spiral  1  and  Spiral  2  missile  improvements,  and  began 
funding  the  development  of  a  Spiral  3  missile  and  a  new  Light 
Weight  Command  Launch  Unit  (CLU).  The  Army  intends 
these  efforts  to  reduce  unit  cost  and  weight,  and  improve 
lethality  against  non-armored  targets. 

•  Early  arena  testing  and  lethality  modeling  of  the  Spiral  2 
missile,  which  includes  a  new  Multi-Purpose  Warhead 
(MPWH),  demonstrated  improved  warhead  fragmentation 
versus  the  legacy  warhead.  LFT&E  of  the  Spiral  2  missile 
will  continue  in  FY16. 

•  Detection,  Recognition,  and  Identification  (DRI) 
characterization  testing  with  Soldiers  demonstrated  that 
gunners  using  Javelin  can  detect,  recognize,  and  lock-on  to 
personnel  in  the  open  out  to  approximately  2,000  meters.  The 
Javelin’s  required  operational  range  against  armored  targets 

is  2,500  meters.  Gunners  required  additional  sources  of  field 
intelligence  to  consistently  identify  threat  from  non-threat 
personnel  beyond  1,000  meters. 

•  DOT&E  and  the  Army  are  in  test  planning  discussions  for  the 
Spiral  3  missile  and  Light  Weight  CLU  developments. 

System 

•  The  Javelin  Close  Combat  Missile  System  -  Medium  (Javelin) 
is  a  man-portable,  fire-and-forget,  precision  anti-tank  missile 
employed  by  dismounted  troops  to  defeat  threat  armored 
combat  vehicles  targets  out  to  2,500  meters.  It  uses  a  lofted 
trajectory  to  deliver  a  top-attack  tandem  shaped  charged 
warhead.  It  also  has  a  direct-fire  mode. 

•  The  Javelin  system  consists  of  a  missile  in  a  disposable  launch 
tube  assembly  and  a  re-usable  CLU.  The  CLU  mechanically 
engages  the  launch  tube  assembly  to  provide  a  stable  platform 
for  firing,  has  day  and  night  sights  for  surveillance  and  target 
acquisition,  and  electronically  interfaces  with  the  missile  for 
target  lock-on  and  missile  launch.  An  operationally-ready 
Javelin  system  weighs  approximately  49.5  pounds. 

•  The  Army  has  planned  four  Javelin  system  improvements  in 
order  to  reduce  unit  cost  and  weight,  and  improve  lethality 
against  non-armored  targets.  These  improvements  are  referred 
to  as  missile  Spiral  1,  2,  3,  and  Light  Weight  CLU. 

-  The  Spiral  1  effort  will  replace  electronic  components 
in  the  control  actuator  section  of  the  missile  for  cost  and 
weight  savings. 


-  The  Spiral  2  effort  will  develop  a  MPWH,  which  uses 
enhanced  fragmentation  to  improve  lethality  against 
non-armored  targets  and  personnel  in  the  open  while 
maintaining  lethality  against  armored  threats. 

-  The  Spiral  3  effort  will  develop  a  new  launch  tube 
assembly  and  replace  electronic  components  in  the 
guidance  section  of  the  missile  for  cost  savings. 

-  The  Light  Weight  CLU  effort  will  develop  an  all  new  CLU 
that  is  smaller  and  lighter  while  maintaining  or  improving 
system  performance. 

Mission 

•  Infantry,  Engineer,  Reconnaissance,  and  Special  Operation 
Forces  within  Army  and  Marine  Corps  ground  maneuver  units 
employ  the  Javelin  to  destroy,  capture,  or  repel  enemy  assault 
through  maneuver  and  firepower. 

•  Service  members  use  the  Javelin  to  destroy  threat  armor 
targets,  light-skinned  vehicles,  and  incapacitate  or  kill  threat 
personnel  within  fortified  positions.  In  recent  conflicts  Javelin 
was  used  primarily  against  enemy  bunkers,  caves,  urban 
structures,  mortar  positions,  snipers,  and  personnel  emplacing 
IEDs. 

Major  Contractors 

•  Raytheon  -  Tucson,  Arizona 

•  Lockheed  Martin  -  Orlando,  Florida 


Activity 

•  In  2015,  the  Javelin  Program  Office  continued  development 
and  testing  of  the  Spiral  1  and  Spiral  2  missile  improvements. 


The  Army  Aviation  and  Missile,  Development,  and 
Engineering  Center,  in  accordance  with  the  DOT&E -approved 
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live  fire  strategy,  conducted  live  fire  arena  tests  of  the  MPWH 
to  characterize  missile  and  warhead  fragmentation.  Static  and 
dynamic  testing,  and  live  warhead  end-to-end  firings  against  a 
range  of  targets  will  continue  in  FY16. 

•  The  Javelin  Program  Office  conducted  two  DR1  tests  with 
Soldiers  in  order  to  characterize  the  ability  of  gunners  using 
only  Javelin  to  detect,  recognize,  identify,  and  lock-on  to 
secondary  targets  (IED  team,  mortar  team,  truck  mounted 
machine  gun)  at  various  ranges  in  both  daylight  and  night 
conditions.  Tests  were  performed  in  February  and  July  2015, 
to  capture  both  cold  and  hot  ambient  surroundings. 

•  DOT&E  and  the  Army  are  in  test  planning  discussions  for  the 
Spiral  3  missile  and  Light  Weight  CLU. 

Assessment 

•  The  preliminary  results  of  available  live  fire  data  indicate  the 
MPWH  exhibits  improved  warhead  fragmentation  versus  the 
legacy  warhead.  A  comprehensive  LFT&E  program  including 
static  and  dynamic  testing,  and  end-to-end  firings  against  a 
variety  of  targets  is  planned  for  FY16  to  verify  the  MPWH 
performance  prior  to  the  production  build  beginning  in  FY17. 

•  Findings  from  the  Javelin  DR1  characterization  test  include: 

-  Military  gunners  using  the  day  and  night  sights  of  the 

Javelin  CLU  without  other  field  intelligence  sources 


(binoculars,  UAV  feed,  etc.),  can  detect  and  recognize 
personnel  in  the  open,  but  cannot  identify  threat  personnel 
from  non-threat  personnel  throughout  the  operational  range 
and  environment. 

-  Military  gunners  were  able  to  maintain  target  acquisition 
throughout  the  process  of  readying  the  Javelin  to  fire  upon 
secondary  targets  identified  as  threats. 

-  Military  gunners  were  able  to  maintain  lock-on  to 
individuals  and  small  groups  out  to  approximately 
2,000  meters.  Maintaining  lock-on  became  more  difficult 
due  to  the  small  signature  presented  by  human  sized  targets 
beyond  2,000  meters. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 

report  for  this  program. 

•  FY15  Recommendation. 

1 .  The  Army  should  conduct  an  operational  test  before 
fielding  Spiral  3  and/or  Light  Weight  CLU  to  confirm  that 
effectiveness  and  suitability  has  not  been  compromised,  and 
compatibility  with  all  fielded  variants  of  the  missile. 
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Joint  Battle  Command  -  Platform  (JBC-P) 


Executive  Summary 

•  During  October  through  November  2014,  the  Army  conducted 
testing  of  JBC-P  6.0  to  verify  fixes  of  deficiencies  noted 
during  the  May  2014  JBC-P  6.0  Multi-Service  Operational 
Test  and  Evaluation  (MOT&E).  JBC-P  continued  to  exhibit 
phantom  Mayday  messages  and  a  new  deficiency  of  delayed 
position  location  information  (PLI)  updates.  The  Army  did 
not  approve  a  full  materiel  release,  but  approved  a  conditional 
materiel  release  to  field  one  Army  brigade  and  continue  testing 
because  these  deficiencies  were  not  corrected. 

•  In  June  2015,  the  Army  conducted  further  testing  of  JBC-P  6.0 
in  conjunction  with  fielding  of  the  first  unit  to  verify  correction 
of  MOT&E  deficiencies.  The  Army  demonstrated: 

-  A  reduction  in  phantom  Mayday  messages  (five  during 
test),  which  were  manageable  through  unit  standard 
operating  procedures. 

-  Correction  of  delayed  PLI  updates. 

-  Proper  representation  of  map  graphics  and  unit  icons. 

The  Army  plans  to  pursue  another  conditional  materiel  release 
to  pursue  further  fielding  and  testing  of  JBC-P  6.0. 

•  The  Army  intends  to  conduct  an  operational  test  of  JBC-P 
Logistics  (Log)  using  an  Army  sustainment  brigade.  The 
Army  will  also  perform  further  cybersecurity  testing  to 
address  deficiencies  noted  during  the  JBC-P  MOT&E. 

System 

•  JBC-P  is  a  networked  battle  command  information  system 
that  enables  units  to  share  near  real-time  friendly  and  enemy 
situational  awareness  information,  operational  maps  and 
graphics,  and  command  and  control  messages. 

•  The  Army  and  Marine  Corps  intend  JBC-P  to  achieve  platform 
level  interoperability  for  ground  vehicles,  dismounted 
Soldiers/Marines,  and  aviation  assets  operating  in  land/littoral 
and  joint  operational  environments. 

•  JBC-P  is  an  upgrade  to  the  Force  XXI  Battle  Command 
Brigade  and  Below  Joint  Capabilities  Release  and  provides  the 
following  improvements: 

-  Tactical  chat  combined  with  chat  room  capability, 
providing  enhanced  collaboration  for  commanders 

-  Improved  mission  command  applications  for  planning  and 
execution 

-  A  more  intuitive  graphical  user  interface  with  an  improved 
map  and  image  display 

-  Enhanced  Blue  Force  situational  awareness  between 
mobile  platforms,  Tactical  Operational  Centers,  and 
dismounted  Soldiers  equipped  with  Nett  Warrior 

-  JBC-P  Log,  which  is  a  logistical  variant  of  JBC-P  that 
supports  sustainment  and  enables  logistics  cargo  tracking 
using  Radio  Frequency  Identification  tags 


1  -  JBC-P  on  Mounted  Family  of  Computer  Systems  (MFoCS) 

2  -  JBC-P  on  Joint  Version  -  5  (JV-5)  Block  2 

3  -  JBC-P  Screen  Shot 

4  -  JBC-P  Log  on  Military  Rugged  Tablet  Plus  (MRT+) 

5  -  JBC-P  on  JV-5  Block  2 

-  Flybrid  capability  to  connect  JBC-P  across  different 
networks  using  its  Network  Services  Gateway  and 
associated  terrestrial  and  satellite  radios 

•  JBC-P  is  fielded  in  both  mobile  and  command  post  versions. 
JBC-P  communications  is  supported  by: 

-  Blue  Force  Tracker  2  satellite  communications  for  mobile 
operations 

-  Tactical  radios  for  connectivity  between  JBC-P-equipped 
vehicles  and  to  support  dismounted  operations 

-  Tactical  Internet  for  command  post  operations 

Mission 

The  Army,  Marine  Corps,  and  Special  Operations  Forces 
commanders  use  JBC-P  to  provide  integrated,  on-the-move, 
near  real-time  battle  command  information  and  situational 
awareness,  from  brigade,  to  maneuver  platform,  to  dismounted 
Soldiers/Marines. 

Major  Contractor 

Software  Engineering  Directorate,  U.S.  Army  Aviation  &  Missile 
Research,  Development  &  Engineering  Center  -  Huntsville, 
Alabama 


JBC-P 
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Activity 

•  In  May  2014,  the  Army  and  Marine  Corps  conducted  a  JBC-P 
software  build  6.0  MOT&E  as  part  of  the  Network  Integration 
Evaluation  (N1E)  14.2  to  support  fielding  decisions  in  both 
Services.  The  MOT&E  was  performed  in  accordance  with  a 
DOT&E-approved  test  plan. 

•  During  October  through  November  2014,  the  Army  conducted 
testing  of  JBC-P  6.0  as  part  of  N1E  15.1  to  verify  fixes  of 
deficiencies  noted  during  the  JBC-P  6.0  MOT&E.  The 
Army  assessed  JBC-P  using  observations,  surveys,  and  focus 
groups  within  the  2nd  Brigade  Combat  Team,  1st  Armored 
Division  operating  under  realistic  mission  conditions  at 

Fort  Bliss,  Texas.  JBC-P  continued  to  demonstrate  phantom 
Mayday  messages  and  exhibited  a  new  deficiency  of  delayed 
PLI  updates.  Since  the  MOT&E  deficiencies  were  not 
corrected,  the  Army  did  not  approve  a  full  materiel  release, 
but  approved  a  conditional  materiel  release  to  field  one  Army 
brigade  and  continue  testing. 

•  In  January  2015,  DOT&E  submitted  a  JBC-P  6.0  MOT&E 
report  with  a  classified  annex  detailing  results  of  testing  during 
NIE  14.2. 

•  In  June  2015,  the  Army  conducted  further  testing  of  JBC-P 
6.0  employing  the  2nd  Infantry  Brigade  Combat  Team, 

3rd  Infantry  Division  operating  under  realistic  mission 
conditions  at  Fort  Stewart,  Georgia,  to  verify  correction  of 
MOT&E  deficiencies.  As  with  NIE  15.1,  the  Army  collected 
observations,  surveys,  and  focus  group  data.  The  Army  plans 
to  pursue  another  conditional  materiel  release  to  pursue  further 
fielding  and  testing  of  JBC-P  6.0. 

•  The  Army  intends  to  conduct  an  operational  test  of  JBC-P 
Log  using  an  Army  sustainment  brigade.  The  Army  will  also 
perform  further  cybersecurity  testing  to  address  deficiencies 
noted  during  the  JBC-P  MOT&E. 

Assessment 

•  In  the  January  2015  JBC-P  MOT&E  report,  DOT&E  assessed 
JBC-P  6.0  as: 

-  Not  operationally  effective  due  to  low  message 
completion  rates,  phantom  Mayday  messages,  inaccurate 
representation  of  blue  force  icons,  and  the  poor 
performance  of  JBC-P  Log. 

-  Not  operationally  suitable  due  to  reliability  that  was  below 
the  Army’s  requirement  for  five  of  seven  JBC-P  hardware 


variants,  deficiencies  in  training  provided  to  leaders  and 
Soldiers,  and  lack  of  a  force  structure  to  support  JBC-P 
Log. 

-  Not  survivable  due  to  cybersecurity  vulnerabilities. 

•  During  FY 1 5,  the  Army  verified  fixes  of  several  JBC-P  6.0 
MOT&E  deficiencies  to  include  demonstrating: 

-  A  reduction  in  phantom  Mayday  messages  (five  during 
test),  which  were  manageable  through  unit  standard 
operating  procedures 

-  Correction  of  delayed  PLI  updates 

-  Proper  representation  of  map  graphics  and  unit  icons 

•  The  following  JBC-P  6.0  MOT&E  deficiencies  remain  to  be 
corrected  and  verified  through  testing: 

-  Cybersecurity  vulnerabilities  identified  in  the  JBC-P 
MOT&E  classified  report 

-  Low  message  completion  rates  of  shared  survivability 
information  (e.g.,  icons  representing  enemy  minefields,  an 
IED,  or  damaged  bridge) 

-  Poor  performance  and  lack  of  force  structure  to  support 
JBC-P  Log 

-  JBC-P  reliability 

•  The  Army  has  not  yet  developed  its  T&E  strategy  for  further 
JBC-P  enhancements  and  the  planned  transition  to  the 
Mounted  Computing  Environment. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  made 
improvements  in  one  of  five  previous  recommendations, 
yet  still  needs  to  improve  JBC-P  message  completion  rates, 
reliability,  JBC-P  Log  force  structure.  Soldier  and  leader 
training,  and  cybersecurity. 

•  FY15  Recommendations.  The  Army  should: 

1.  Continue  to  correct  JBC-P  6.0  MOT&E  deficiencies  and 
conduct  developmental  and  operational  testing  to  verify 
fixes  of  MOT&E  deficiencies  to  include  cybersecurity 
vulnerabilities,  low  message  completion  rates,  reliability, 
and  JBC-P  Log  performance. 

2.  Improve  JBC-P  leader  and  Soldier  training. 

3.  Update  the  JBC-P  Test  and  Evaluation  Master  Plan 
to  include  testing  to  verify  fixes  and  future  JBC-P 
enhancements. 
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Joint  Light  Tactical  Vehicle  (JLTV) 
Family  of  Vehicles  (FoV) 


Executive  Summary 

•  In  August  2015,  DOT&E  published  the  Joint  Light  Tactical 
Vehicle  (JLTV)  Operational  Assessment  and  classified  LFT&E 
reports  to  support  the  Defense  Acquisition  Board  JLTV 
Milestone  C  decision. 

•  The  Defense  Acquisition  Executive  approved  the  JLTV 
program  to  enter  Milestone  C  low-rate  initial  production  in 
August  2015. 

•  The  Army  awarded  the  JLTV  low-rate  initial  production 
contract  to  Oshkosh  Corporation  in  August  2015. 

•  In  September  2015,  Lockheed  Martin  Corporation  protested 
the  Army’s  decision  to  award  the  JLTV  contract  to  Oshkosh 
Corporation.  The  General  Accountability  Office  dismissed 
the  protest  in  December  2015  because  Lockheed  indicated  it 
would  take  the  matter  to  the  Court  of  Federal  Claims. 

System 

•  The  JLTV  Family  of  Vehicles  (FoV)  is  the  Marine  Corps  and 
Army  partial  replacement  for  the  High  Mobility  Multi-purpose 
Wheeled  Vehicle  (HMMWV)  fleet.  The  Services  intend  the 
JLTV  to  provide  increased  crew  protection  against  IEDs  and 
underbody  attacks,  improved  mobility,  and  higher  reliability 
than  the  HMMWV. 

•  The  JLTV  FoV  consists  of  two  vehicle  categories:  the  JLTV 
Combat  Tactical  Vehicle,  designed  to  seat  four  passengers, 
and  the  JLTV  Combat  Support  Vehicle,  designed  to  seat 
two  passengers. 

•  The  JLTV  Combat  Tactical  Vehicle  has  a  3,500-pound  payload 
and  three  mission  package  configurations: 

-  Close  Combat  Weapons  Carrier  Vehicle 

-  General  Purpose  Vehicle 

-  Heavy  Guns  Carrier  Vehicle 

•  The  JLTV  Combat  Support  Vehicle  has  a  5,100-pound  payload 
and  one  mission  package  configuration: 

-  Utility  Prime  Mover  that  can  accept  a  Shelter  Carrier. 

-  Utility  Prime  Mover 

•  JLTV  vehicles  are  equipped  with  vendor-unique  solutions 
intended  to  significantly  improve,  relative  to  the  HMMWV, 
crew  protection  against  the  effects  of  small  arms,  fragments, 
and  underbody  and  underwheel  blast  loading  from  mines  and 
IEDs.  These  include  the  design  of  the  vehicle  underbody 
hidl  structure,  energy-attenuating  seats,  and  floor  specifically 
designed  to  mitigate  blast  loading  to  the  occupants. 

•  JLTV  vehicles  are  equipped  with  two  separate  armor  levels: 
the  A-kit,  or  base  vehicle,  which  is  intended  for  use  in 
low-threat  environments,  and  the  B-kit,  an  add-on  armor  kit, 
for  additional  force  protection  in  the  intended  deployment 
configuration  but  at  the  cost  of  additional  weight. 


Oshkosh  Corporation  JLTV 


Lockheed  Martin 
Systems  JLTV 


AM  General  JLTV 


Mission 

•  Military  units  employ  JLTV  as  a  light,  tactical-wheeled 
vehicle  to  support  all  types  of  military  operations.  JLTVs  are 
used  by  airborne,  air  assault,  amphibious,  light,  Stryker,  and 
heavy  forces  as  reconnaissance,  maneuver,  and  maneuver 
sustainment  platforms. 


JLTV  FoV 
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•  Small  ground  combat  units  will  employ  JLTV  in  combat  •  Tested  Contractors: 

patrols,  raids,  long-range  reconnaissance,  and  convoy  escort.  -  Lockheed  Martin  Corporation  -  Grand  Prairie,  Texas 

-  AM  General  -  South  Bend,  Indiana 

Major  Contractors 

•  Oshkosh  Corporation,  Oshkosh,  Wisconsin 


Activity 

•  In  April  2014,  the  Army  and  Marine  Corps  units  conducted 
air  assault  missions  during  developmental/operational  at 
Aberdeen  Proving  Ground,  Maryland,  using  CH-47F  and 
CH-53E  helicopters.  The  Marine  Corps  unit  conducted 
amphibious  assault  missions  at  Joint  Base  Little  Creek, 

Fort  Story,  Virginia,  using  Landing  Craft  Utility  ships. 

•  In  October  2014,  the  Army  Test  and  Evaluation  Command 
completed  the  Engineering  Manufacturing  and  Development 
ballistic  phase  of  the  LFT&E  program  of  all  three 
vendor-provided  JLTV  prototypes,  which  included: 

-  Armor  coupon  testing  against  the  medium  machine  gun, 
as  well  as  fragments  from  side  and  underbody  IEDs,  and 
overheard  artillery  to  assess  if  the  ballistic  protection 
performance  of  vendor  armor  solutions  the  JLTV 
requirements. 

-  Ballistic  structure  testing  on  both  the  base  A-kit  structure 
and  the  up-armored  B-kit  structure.  This  consisted  of 
exploitation  testing  to  determine  the  vulnerability  of 
unique  armor  features  on  each  JLTV  prototype,  as  well  as 
blast-fragmentation  IED  testing  conducted  to  determine  the 
structural  vulnerability,  resistance  to  penetration,  and  force 
protection  provided  by  JLTV  prototypes. 

-  System-level  testing  against  underbody  mines  and 
IEDs,  underwheel  mines,  side  IEDs,  rocket-propelled 
grenades,  and  explosively-formed  penetrators  to  assess  the 
vendors’  compliance  with  force  protection  requirements, 
the  vulnerability  of  the  vehicle  design,  and  vehicle 
recoverability  post-event. 

•  In  November  2014,  the  Army  Test  and  Evaluation 
Command  and  Marine  Corps  Operational  Test  and 
Evaluation  Agency  conducted  the  JLTV  Limited  User  Test 
(LUT)  at  Fort  Stewart,  Georgia,  in  accordance  with  the 
DOT&E-approved  test  plan.  The  Army  test  unit  completed 
three,  96-hour  scenarios  and  the  Marine  Corps  test  unit 
completed  one,  96-hour  scenario  at  the  operational  tempo 
consistent  with  the  JLTV  Operational  Mode  Summary /Mission 
Profile. 

•  The  Joint  Requirements  Oversight  Council  approved  the  JLTV 
Capability  Production  Document  in  November  2014. 

•  The  JLTV  Program  Office  completed  development  of  the  JLTV 
FoV  Milestone  C  Test  and  Evaluation  Master  Plan  (TEMP)  to 
reflect  the  T&E  activities  for  the  production  and  deployment 
phase  in  May  2015.  The  Army  did  not  submit  the  Milestone  C 
JLTV  TEMP  for  OSD  approval  prior  to  the  Milestone  C. 

•  In  August  2015,  DOT&E  submitted  the  JLTV  Operational 
Assessment  and  classified  LFT&E  reports  to  support  the 
Defense  Acquisition  Board  JLTV  Milestone  C  decision. 

126  JLTV  FoV 


•  The  Defense  Acquisition  Executive  approved  the  JLTV 
program  to  enter  Milestone  C  low-rate  initial  production  in 
August  2015. 

The  Army  awarded  the  JLTV  low-rate  initial  production 
contract  to  Oshkosh  Corporation  in  August  2015. 

•  In  September  2015,  Lockheed  Martin  Corporation  protested 
the  Army’s  decision  to  award  the  JLTV  contract  to  Oshkosh 
Corporation.  The  General  Accountability  Office  dismissed 
the  protest  in  December  2015  because  Lockheed  indicated  it 
would  take  the  matter  to  the  Court  of  Federal  Claims. 

Assessment 

•  Based  on  the  LUT,  the  JLTV  FoV  provides  enhanced 
protection  and  retains  the  up-armored  HMMWV  (UAH) 

FoV  capabilities  necessary  for  Army  and  Marine  units  to 
accomplish  tactical  and  combat  missions. 

-  Platoons  equipped  with  the  Oshkosh  JLTVs  accomplished 
15  out  of  24  missions  similar  to  the  platoon  equipped  with 
the  UAHs. 

-  Platoons  equipped  with  the  AM  General  JLTVs 
accomplished  13  out  of  24  missions. 

-  Platoons  equipped  with  the  Lockheed  Martin  JLTVs 
accomplished  12  out  of  24  missions. 

-  The  majority  of  failed  platoon  missions  were  attributed  to 
combat  losses  for  Oshkosh  and  Lockheed  Martin  JLTVs. 

-  Platoons  equipped  with  the  AM  General  JLTVs  and  the 
UAHs  experienced  less  combat  losses  against  the  Opposing 
Force  during  missions. 

-  Platoons  equipped  with  AM  General  JLTVs  experienced 
reliability  failures  on  nine  missions  that  slowed  the  unit’s 
pace  and  degraded  mission  performance. 

•  The  JLTVs  have  similar  mobility  capabilities  to  the  UAH 
without  the  Fragmentation  Kit  5.  During  the  LUT,  units 
equipped  with  JLTVs  experienced  delays  in  maneuvering 
while  awaiting  adjustment  of  the  vehicle  suspension 
and  the  Central  Tire  Inflation  System  (CTIS).  The  slow 
suspension  and  CTIS  adjustment  times  affected  the  Army  and 
Marine  Corps  units’  ability  to  quickly  react  to  changes  in  the 
tactical  situation  and  in  some  LUT  missions  increased  the 
units’  susceptibility  to  threats. 

•  Oshkosh  JLTVs  had  improved  mission  reliability  over 
the  UAH  and  demonstrated  7,05 1  Mean  Miles  Between 
Operational  Mission  Failure  (MMBOMF)  versus  its 
operational  requirement  of  2,400  MMBOMF.  The  UAH 
demonstrated  2,968  MMBOMF. 
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-  AM  General  JLTVs  had  less  mission  reliability  versus  the 
UAH  and  demonstrated  526  MMBOMF. 

-  Lockheed  Martin  JLTVs  had  less  mission  reliability  versus 
the  UAH  and  demonstrated  1,271  MMBOMF. 

•  Marine  Corps  units  equipped  with  JLTVs  have  enhanced 
capabilities  to  accomplish  air  assault  missions  over  the  UAH. 
Since  the  CH-53E  has  the  capability  to  lift  JLTVs  with  armor, 
units  have  better  protected  maneuver  capabilities  to  counter 
threat  activities  at  the  Landing  Zone  compared  to  units 
equipped  with  the  UAH. 

•  Army  units  cannot  accomplish  air  assault  missions  with 
JLTVs  with  B-kit  armor  because  the  vehicle’s  gross  weight 
exceeds  the  external  lift  capability  of  the  CH-47F  helicopter. 
The  vendors’  JLTVs  with  add-on  B-kit  armor  weigh  between 
18,000  and  22,000  pounds. 

•  Marine  Corps  units  equipped  with  the  JLTVs  demonstrated 
the  ability  to  conduct  amphibious  assault  missions  during 
developmental/operational  testing.  JLTVs  are  slower  to  load, 
prepare  for  fording  and  transition  to  maneuver  ashore  than  the 
UAH  due  to  their  larger  size  and  movement  delays  awaiting 
adjustment  of  the  vehicle  suspension  and  tire  pressure. 

•  The  JLTVs  do  not  have  sufficient  capability  to  carry  mission 
equipment,  supplies,  and  water  for  extended  mission  beyond 
one  day  of  supply.  This  limits  the  type  and  duration  of 
missions  for  which  JLTV  is  effective.  Units  operating  for  long 
duration  will  require  additional  trailers  or  vehicles  to  sustain 
operations. 

•  The  JLTV  Utility  variants  do  not  have  the  capability  to  carry 
troops  like  the  UAH  Cargo/Troop  Carrier.  This  is  not  a  current 
JLTV  requirement.  These  variants  have  no  seats,  no  head 
room,  and  no  underbody  crew  protection  in  the  rear  cargo 
area.  Army  and  Marine  Corps  units  employ  the  HMMWV 
Cargo/Troop  Carrier  to  carry  troops  required  for  combat  and 
combat  support  missions. 

•  The  JLTVs  suffered  from  poor  command,  control,  and 
communication  equipment  integration  by  the  vendor  affecting 
the  unit  commander’s  ability  to  command  and  control 


platoons,  maintain  situational  awareness,  and  complete 
mission  tasks  during  the  LUT. 

•  Due  to  small  rear  windows  and  blind  spots  around  the 
vehicles,  the  JLTVs  did  not  provide  the  Army  and  Marine 
Corps  crews  with  sufficient  visibility  throughout  the  missions. 
Crews  shared  information  of  potential  threats,  movements, 
and  activities  while  moving  to  maintain  shared  situational 
awareness  for  unit  security. 

•  Both  the  Oshkosh  and  Lockheed  Martin  JLTV  prototypes 
met  all  threshold  force  protection  requirements  and  some 
objective-level  requirements.  Both  of  these  prototypes 
provide  protection  superior  to  the  up-armored  HMMWV  and 
similar  to  the  MRAP  All-Terrain  Vehicle  (M-ATV)  without 
the  Underbody  Improvement  Kit  across  the  spectrum  of 
tested  threats.  Oshkosh  implemented  lessons  learned  from 
the  M-ATV  program  into  their  JLTV  prototypes  to  provide 
M-ATV  levels  of  underbody  protection  on  a  lighter  vehicle. 
Lockheed  Martin’s  prototype  provided  protection  on  par  with 
the  M-ATV.  However,  AM  General’s  prototype  would  require 
a  significant  redesign  to  meet  threshold  force  protection 
requirements.  Detailed  findings  on  the  performance  of  the 
vehicle  underbody  hull  structure,  armor,  energy-attenuating 
seat  and  floor  designs,  and  their  aggregate  impact  on 
survivability  against  the  threshold  and  other  operationally 
relevant  threats,  are  outlined  in  DOT&E’s  classified  JLTV 
LFT&E  report. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  addressed 
all  previous  recommendations. 

•  FY 15  Recommendations.  The  program  should: 

1 .  Develop  a  plan  to  address  recommendations  identified  in 
DOT&E’s  Operational  Assessment  and  classified  LFT&E 
reports  before  production. 

2.  Submit  the  Milestone  C  JLTV  TEMP  prior  to  start  of 
government  developmental  testing. 
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Ml  09  Family  of  Vehicles  (FoV) 
Paladin  Integrated  Management  (PIM) 


Executive  Summary 

•  The  Army  continued  multiple  phases  of  the  Ml 09  Family 
of  Vehicles  (FoV)  Paladin  Integrated  Management  (PIM) 
LFT&E  program  at  Aberdeen  Proving  Ground,  Maryland,  to 
include  testing  of  the  underbody  1ED  protection  kit.  Automatic 
Fire  Extinguishing  Systems  (AFES),  ballistic  protection  of 
modified  armored  areas,  and  system  response  to  simulated 
damaged  sub-systems.  In  FY15,  the  Army: 

-  Validated  modifications  to  the  Self  Propelled  Howitzer’s 
(SPH)  Threshold  1  (Tl)  and  Threshold  2  (T2)  armor 
systems,  including  those  made  to  address  vulnerable  areas 
identified  in  early  testing.  Most  of  the  modified  armored 
areas  provide  protection  against  requirement  threats. 

-  Integrated  and  tested  changes  to  the  crew  compartment 
AFES  in  the  Carrier,  Ammunition  Tracked  (CAT).  These 
measures  mitigated  the  AFES  deficiency  identified  in 
earlier,  FY14  testing. 

-  Verified  that  the  SPH  has  the  potential  to  provide 
underbody  IED  protection  against  the  requirement  threat 
and  the  objective  level  threat  when  equipped  with  the 
underbody  blast  kit.  At  this  time,  the  Army  does  not  intend 
to  equip  the  SPH  or  CAT  vehicles  with  the  underbody  kit. 

•  The  Army  did  not  address  the  deficiencies  identified  in  fire 
survivability  testing  of  the  SPH  crew  compartment  AFES  and 
should  take  measures  to  reduce  vulnerability  to  fires  in  the 
SPH  crew  compartment. 

•  The  Army  began  full-up  system-level  testing  of  the  SPH  and 
CAT  resupply  vehicle  in  1QFY16. 

System 

•  The  M109  FoV  PIM  consists  of  two  vehicles:  the  SPH  and 
CAT  resupply  vehicle. 

-  The  M109A7  SPH  is  a  tracked,  self-propelled  155  mm 
howitzer  designed  to  improve  sustainability  over  the 
legacy  M109A6  howitzer  fleet.  The  full-rate  production 
howitzers  will  have  a  modified  M109A6  turret  with  a 
high-voltage  electrical  system  and  a  modified  Bradley 
Fighting  Vehicle  chassis,  power  train,  and  suspension.  The 
M109A7  does  not  include  any  upgrades  to  the  cannon.  A 
crew  of  four  Soldiers  operates  the  SPH  and  can  use  it  to 
engage  targets  at  ranges  of  22  kilometers  using  standard 
projectiles  and  30  kilometers  using  rocket-assisted 
projectiles. 

-  The  M992A3  CAT  supplies  the  SPH  with  ammunition. 

The  full-rate  production  ammunition  carriers  will  have 
a  chassis  similar  to  the  SPH.  The  ammunition  carriers 
are  designed  to  carry  12,000  pounds  or  98  rounds  of 
ammunition  in  various  configurations.  A  crew  of  four 
Soldiers  operates  the  CAT. 


•  The  Army  will  equip  the  SPH  and  CAT  with  two  armor 
configurations  to  meet  two  threshold  requirements  for  force 
protection  and  survivability  -  Threshold  1  (Tl)  and  Threshold 
2  (T2). 

-  The  base  Tl  armor  configuration  is  integral  to  the  SPH  and 
CAT.  The  T2  configuration  is  intended  to  meet  protection 
requirements  beyond  the  Tl  threshold  with  add-on  armor 
kits. 

-  The  Army  plans  to  employ  PIM  vehicles  in  the  Tl 
configuration  during  normal  operations  and  will  equip  the 
SPH  and  CAT  with  T2  add-on  armor  kits  during  combat 
operations. 

•  The  Army  designed  an  underbody  kit  to  determine  the 
potential  protection  an  SPH  and  CAT  could  provide  against 
IEDs  similar  to  those  encountered  in  Iraq  and  Afghanistan. 

The  Army  purchased  five  underbelly  kits  for  test  purposes.  At 
this  time,  the  Army  does  not  intend  to  equip  the  SPH  or  CAT 
with  the  underbody  kit. 

•  The  Army  intends  to  employ  the  M109  FoV  as  part  of  a  Fires 
Battalion  in  the  Armored  Brigade  Combat  Team  and  Artillery 
Fires  Brigades,  with  the  capability  to  support  any  Brigade 
Combat  Team. 

•  The  Army  plans  to  field  up  to  557  sets  of  the  M109  FoV  with 
full-rate  production  planned  for  FY17. 

Mission 

Commanders  employ  field  artillery  units  equipped  with  the 
M109  FoV  to  destroy,  defeat,  or  disrupt  the  enemy  by  providing 
integrated,  massed,  and  precision  indirect  fire  effects  in  support 
of  maneuver  units  conducting  unified  land  operations. 

Major  Contractor 

BAE  Systems  -  York,  Pennsylvania 
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Activity 

•  In  FY15,  the  Army  continued  LFT&E  of  the  M109  FoV 
PIM  at  Aberdeen  Proving  Ground,  Maryland,  to  include  the 
following: 

-  Exploitation  testing  on  the  SPH  5A  to  validate  armor 
modifications.  The  Army  validated  modifications  to  the 
SPH’s  T1  and  T2  armor  systems,  including  those  made  to 
address  vulnerable  areas  identified  in  early  testing. 

-  Fire  survivability  testing.  AFES  testing  in  FY 14  identified 
system  deficiencies  with  the  crew  compartment  AFES 

in  both  the  SPH  and  CAT.  To  improve  survivability,  the 
Army  made  changes  to  the  crew  compartment  AFES  in  the 
CAT  vehicle.  The  Army  integrated  and  challenged  these 
changes  in  test.  However,  the  Army  did  not  address  the 
deficiencies  identified  in  fire  survivability  testing  of  the 
SPH  crew  compartment  AFES. 

-  Underbody  blast  testing  against  the  “SPH  5A”,  a 
high-fidelity  SPH  prototype.  The  testing  included 

two  events  with  and  without  an  objective-level  underbody 
blast  kit. 

-  Controlled  damage  experimentation  on  selected  subsystem 
(such  as  the  high  voltage  electrical  system)  to  determine 
the  consequences  of  various  types  of  damage. 

•  The  Army  conducted  all  testing  in  accordance  with 
DOT&E-approved  test  plans. 

•  The  Army  began  full-up  system-level  testing  of  the  M109  SPH 
and  CAT  resupply  vehicle  in  1QFY16. 

Assessment 

•  Over  the  course  of  the  LFT&E  program,  the  Program  Office 
has  taken  considerable  action  to  correct  deficiencies  identified 
in  early  testing  and  to  validate  associated  fixes. 


-  During  armor  exploitation  testing,  most  of  the  modified 
armored  areas  demonstrated  that  they  provide  protection 
against  Key  Performance  Parameter  threats. 

-  Changes  to  the  crew  compartment  AFES  in  the  CAT 
mitigate  the  deficiency  identified  in  early  testing  and 
reduce  the  CAT’s  vulnerability  to  fires. 

•  The  crew  compartment  AFES  in  the  SPH  was  designed  to 
only  protect  a  localized  area  in  the  compartment  and  therefore 
remains  deficient.  The  system  should  be  redesigned  to 
improve  SPH  survivability  to  fires. 

•  The  Army  verified  that  the  base  SPH  has  the  potential  to 
provide  underbody  IED  protection  against  the  requirement 
blast  threat  and  the  objective  level  threat  when  equipped  with 
the  underbody  blast  kit.  Additional  underbody  blast  testing 
of  the  SPH  and  CAT  is  required  to  quantify  the  benefit  of 
equipping  both  platforms  with  the  kit. 

Recommendations 

•  Status  of  Previous  Recommendations.  lnFY14,  DOT&E 
recommended  the  Army  correct  deficiencies  identified  in  fire 
survivability  testing.  In  FY15,  the  Army  made  design  changes 
to  mitigate  the  deficiencies  in  the  CAT’s  crew  compartment 
AFES  and  validated  those  changes  in  test.  The  Army  did  not 
incorporate  changes  to  address  the  deficiencies  in  the  SPH’s 
crew  compartment  AFES. 

•  FY15  Recommendation. 

1.  The  Army  should  correct  the  deficiencies  in  the  SPH’s  crew 
compartment  AFES  and  validate  those  fixes  in  test. 
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M270A1  Multiple  Launch  Rocket  System  (MLRS) 
Improved  Armored  Cab  (IAC) 


Executive  Summary 

•  In  December  2008,  the  Army  directed  that  chassis 
improvement  requirements  identified  in  the  approved 
High  Mobility  Rocket  System  (H1MARS)  Operational 
Requirements  Document  be  incorporated  in  the  M270A1 
Multiple  Launch  Rocket  System  (MLRS)  launcher.  This 
resulted  in  the  development  of  an  Improved  Armor  Cab  (IAC) 
to  replace  existing  M270A1  cabs,  improved  armor  protection, 
and  the  addition  of  mine  protective  seating.  The  Army  wanted 
to  provide  greater  crew  protection  across  all  fielded  MLRS 
launchers  comparable  to  that  provided  by  the  HIMARS. 

•  The  M270A1  IAC  is  designed  to  protect  the  crew  against 
various  direct  fire,  mine  blast,  and  underbody  fragmenting 
improvised  explosive  device  threats. 

•  The  Army  completed  the  M2 70 A 1  IAC  LFT&E  program  in 
September  2015.  DOT&E’s  preliminary  analyses  of  the  armor 
coupon  and  exploitation  test  data,  and  system-level  and  full-up 
system-level  live  fire  test  data,  indicate  that  the  IAC  protects 
the  crew  against  the  specified  threats. 

System 

•  The  M270A1  MLRS  is  a  tracked,  indirect  fire,  field  artillery 
system  capable  of  firing  all  rockets  and  missiles  in  the  MLRS 
Family  of  Munitions. 

•  The  M270A1  IAC  upgrades  the  crew  protection  of  the 
currently-fielded  M270A1  MLRS. 

•  This  improvement  in  crew  protection  is  intended  to  protect 
the  M270A1  crew  against  a  variety  of  threats,  including  direct 
fire,  mine  blast,  and  underbody  fragmenting  IED  events. 

•  The  upgrades  include  cab  and  hull  modifications  to  improve 
occupant  survivability,  and  suspension  modifications  to 
accommodate  the  vehicle’s  increased  weight. 

•  This  crew  protection  upgrade  provides  M270A1  crewmembers 
the  same  protection  as  currently  provided  to  crews  operating 


the  wheeled  HIMARS.  Both  HIMARS  and  MLRS  operate 
in  a  similar  manner  and  fire  identical  rocket  and  missile 
munitions. 

Mission 

Commanders  will  employ  units  equipped  with  the  improved 
M270A1  launcher  to  provide  medium-range  field  artillery 
rocket  and  long-range  missile  fires  in  support  of  ground  forces 
to  destroy,  neutralize,  or  suppress  the  enemy,  in  accordance  with 
applicable  tactics,  techniques,  and  procedures. 

Major  Contractors 

•  Lockheed  Martin  Missile  and  Fire  Control  Division 
Grand  -  Prairie,  Texas  (system  integrator) 

•  BAE  Ground  Systems  Division  -  Santa  Clara,  California  (cab 
structure,  chassis  armor) 


Activity 

•  The  M270A1  IAC  program  is  being  conducted  per  a 

December  2008  memorandum  from  the  Office  of  the  Deputy 
Chief  of  Staff,  G-3/5/7  requiring  synchronization  of  protection 
requirements  across  all  MLRS  launchers.  The  M270A1 
IAC  will  provide  M270A1  crewmembers  with  protection 
comparable  to  that  provided  by  the  HIMARS.  The  IAC  is 
an  Engineering  Change  Proposal  to  the  M270A1  and  does 
not  impact  the  launcher’s  system  performance  or  the  tactics, 
techniques,  or  procedures  used  during  tactical  operations. 


•  In  January  2015,  DOT&E  approved  the  Army’s  test  plan  for 
the  live  fire  testing  of  the  M270A1  IAC.  The  Army  will  use 
the  results  of  live  fire  testing  to  determine  whether  the  IAC 
provides  the  required  crew  protection  against  the  required 
operationally  relevant  conventional  ballistic  threats. 

•  From  January  through  August  2015,  the  Army  conducted 

six  system-level  and  three  full-up  system-level  live  fire  events 
using  a  production-representative  IAC  against  specified 
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threats.  These  threats  included  mines,  fragmenting  IEDs,  and 
an  explosively-formed  projectile. 

•  The  Army  also  conducted  armor  coupon  and  exploitation 
testing  against  several  threats.  Coupon  testing  produced 
data  that  provided  general  vulnerability  insights  as  well 
as  specific  information  to  support  complex  vulnerability 
modeling.  Exploitation  testing  was  conducted  on  a 
production-representative  cab  to  obtain  data  on  the  ballistic 
protection  capabilities  of  the  cab  at  locations  such  as  seams, 
edges,  and  bolts  on  the  cab.  The  Army  completed  its 
exploitation  testing  in  September  2015. 

•  In  2QFY16,  DOT&E  intends  to  submit  a  classified  live  fire 
report  for  the  M270A1  IAC  upon  completion  of  testing  and 
analysis  of  test  results. 


Assessment 

Analysis  of  the  test  data  is  ongoing.  DOT&E ’s  preliminary 
analysis  of  the  armor  coupon  and  exploitation  test  data,  and 
system-level  and  full-up  system-level  live  fire  test  data  indicate 
that  the  improved  armored  cab  protects  the  crew  against  the 
specified  threats.  The  classified  DOT&E  report  will  provide  a 
detailed  survivability  evaluation  of  the  M270A1  IAC. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY 15  Recommendations.  There  are  currently  no 
recommendations,  as  analysis  of  test  data  is  ongoing.  DOT&E 
will  include  any  recommendations  in  its  final  live  fire  report. 
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M829A4  (formerly  M829E4)  Armor  Piercing,  Fin 
Stabilized,  Discarding  Sabot  -  Tracer  (APFSDS-T) 


Executive  Summary 

•  The  M829A4  120  mm  cartridge  is  a  line-of-sight  kinetic 
energy  cartridge  designed  for  the  Abrams  M1A2  System 
Enhancement  Program  version  3  (SEPv3)  Main  Battle  Tank 
(MBT). 

•  In  FY15,  the  Army  implemented  changes  to  the  M829A4 
cartridge  production  processes,  after  multiple  test-fix-test 
iterations  to  address  in-bore  structural  failures  observed  in 
early  testing. 

•  In  F ebruary  2015,  the  Army  conducted  Verification  #2  testing 
at  Yuma  Proving  Ground,  Arizona,  in  order  to  validate  that  the 
newly  configured  cartridge  met  reliability  requirements. 

•  In  May  2015,  the  Army  completed  the  seven  remaining  live 
fire  test  events,  representing  various  engagement  scenarios 
against  threat  target  surrogates. 

•  In  October  2015,  the  Army  type-classified  the  M829E4 
cartridge  as  the  M829A4,  establishing  the  cartridge’s 
acceptability  for  Army  use  and  enabling  the  Program  Office 
to  begin  official  planning  for  production  and  fielding  of  the 
cartridge. 

•  In  December  2015,  DOT&E  submitted  the  classified  M829A4 
combined  OT&E/LFT&E  report  and  assessed  the  following: 

-  The  M829A4  cartridge  is  lethal  and  operationally  effective. 
The  cartridge’s  lethality  and  operational  effectiveness  are 
dependent  on  engagement  conditions  that  are  discussed  in 
the  classified  combined  OT&E/LFT&E  report. 

-  The  M829A4  cartridge  is  suitable.  It  met  its  reliability 
requirement  as  a  point  estimate. 

-  In  comparison  to  previously  fielded  kinetic  energy 
cartridges,  the  M829A4  cartridge  is  not  expected  to 
increase  the  vulnerability  associated  with  the  stowed 
ammunition  in  the  Abrams  M1A2  SEPv3  MBT,  if  engaged 
by  an  overmatching  threat. 

•  The  Army’s  Full-Rate  Production  decision  was  in 
December  2015. 

System 

•  The  M829A4  120  mm  cartridge  is  a  line-of-sight  kinetic 
energy  cartridge  designed  for  the  Abrams  M1A2  SEPv3  MBT. 
It  is  the  materiel  solution  for  the  Abrams’  lethality  capability 
gap  against  threat  vehicles  equipped  with  third-generation 
explosive  reactive  armor. 

•  The  M829A4  cartridge  is  an  Armor-Piercing,  Fin-Stabilized, 
Discarding  Sabot,  with  Tracer  cartridge  consisting  of  a 
depleted  uranium  long-rod  penetrator  with  a  three-petal 
composite  sabot. 


Ammunition  Data  Link 
(ADL)  interface  rings 
on  base  of  the 
M829A4. 


•  The  flight  projectile  includes  a  low-drag  fin  with  a  tracer, 
windshield,  and  tip  assembly. 

•  The  propulsion  system  of  the  M829A4  cartridge  is  a 
combustible  cartridge  case  similar  to  that  of  the  currently 
fielded  suite  of  Abrams’  120  mm  tank  cartridges. 

•  The  M829A4  has  comparable  characteristics  to  its 
predecessor,  the  M829A3,  in  length,  weight,  and  center  of 
gravity.  The  visible  difference  between  the  two  cartridges  is 
the  Ammunition  Data  Link  (ADL)  interface  rings  on  the  base 
of  the  M829A4.  The  rings  serve  as  the  interface  between 
the  Abrams’  fire  control  system  and  the  M829A4.  The  ADL 
enables  the  Abrams’  fire  control  system  to  send  information  to 
the  M829A4. 

Mission 

Commanders  will  employ  units  equipped  with  Abrams  MBTs 
that  use  the  M829A4  120  mm  cartridge  to  defeat  current  and 
projected  threat  tanks  that  are  equipped  with  third  generation 
explosive  reactive  armor  and  active  protection  systems.  The 
Army  intends  the  M829A4  to  provide  lethality  beyond 
its  predecessor,  the  M829A3,  enhancing  the  Joint  Forces 
Commander’s  capability  to  conduct  decisive  operations  during 
Unified  Land  Operations. 

Major  Contractor 

Alliant  Techsystems  Inc.  (ATK)  -  Plymouth,  Minnesota 
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Activity 

•  The  Army  proposed  changes  in  design  configuration  and 
production  processes  to  address  in-bore  structural  failures 
observed  in  FY14  testing  and  improve  the  cartridge’s 
reliability.  After  multiple  test-fix-test  iterations  and  failure 
analyses,  the  Program  Office  implemented  four  production 
process  changes. 

•  In  F ebruary  2015,  the  Army  conducted  Verification  #2  testing 
at  Yuma  Proving  Ground,  Arizona,  to  validate  that  the  newly 
configured  cartridge  met  reliability  requirements. 

•  The  outcome  of  the  second  phase  of  verification  testing  in 
February  2015  enabled  the  Army  to  resume  production  of  the 
cartridge  and  First  Article  Acceptance  Testing. 

•  In  May  2015,  the  Army  completed  the  seven  remaining  live 
fire  test  events,  representing  various  engagement  scenarios 
against  multiple  threat  target  surrogates. 

•  DOT&E  assessed  data  resulting  from  ammunition 
vulnerability  testing  conducted  in  FY14. 

•  The  Army  type-classified  the  M829E4  cartridge  in 
October  2015  as  the  M829A4  establishing  the  cartridge’s 
acceptability  for  Army  use.  This  enabled  the  Program  Office 
to  begin  official  planning  for  production  and  fielding  of  the 
cartridge. 

•  DOT&E  submitted  the  classified  M829A4  combined 
OT&E/LFT&E  report  in  December  2015. 

•  The  Army’s  Full-Rate  Production  decision  was  in 
December  2015. 


Assessment 

•  DOT&E  assessed  the  following  in  the  December  2015 
OT&E/LFT&E  report: 

-  The  M829A4  cartridge  is  lethal  and  operationally  effective. 
The  cartridge’s  lethality  and  operational  effectiveness  are 
dependent  on  engagement  conditions  that  are  discussed  in 
the  classified  combined  OT&E/LFT&E  report. 

The  M829A4  cartridge  is  suitable.  It  met  its  reliability 
requirement  as  a  point  estimate. 

-  If  engaged  by  an  overmatching  threat,  the  M829A4 
cartridge  is  not  expected  to  increase  the  vulnerability 
associated  with  the  stowed  ammunition  in  the  Abrams 
M1A2  SEPv3  MBT  in  comparison  to  previously  fielded 
kinetic  energy  cartridges. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed  all 
previous  recommendations. 

•  FY15  Recommendation. 

1 .  The  Army  should  address  the  recommendations  detailed  in 
the  classified  December  2015  Combined  OT&E/LFT&E 
report. 
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Mid-Tier  Networking  Vehicle  Radio  (MNVR) 


Executive  Summary 

•  From  April  through  May  2015,  the  Army  conducted  a  Mid-Tier 
Networking  Vehicle  Radio  (MNVR)  Limited  User  Test  (LUT) 
as  part  of  the  Network  Integration  Evaluation  (NIE)  15.2  at 
Fort  Bliss,  Texas,  in  accordance  with  a  DOT&E -approved  test 
plan. 

•  DOT&E’s  evaluation  based  on  the  2015  MNVR  LUT  is: 

-  MNVR  enhanced  the  unit’s  mid-tier  network 
when  operating  within  a  full  network,  i.e.  satellite 
communications  were  available. 

-  In  a  reduced  satellite  network  environment,  the 
demonstrated  MNVR  Wideband  Networking  Waveform 
(WNW)  network  message  completion  rate  was  less  than 
76  percent,  which  does  not  meet  the  Army’s  MNVR 
requirement  of  90  percent  at-the-halt  and  85  percent 
on-the-move. 

-  The  MNVR  WNW  network  experienced  faults  that 
prevented  4  of  12  battalion  MNVRs  from  sending  or 
receiving  any  data  for  extended  time  periods  (up  to 
36  hours). 

The  unit  deployed  security  for  MNVR  retransmission 
vehicles  (necessary  to  provide  network  area  coverage). 

This  security  requirement  reduced  the  unit’s  combat  power 
by  up  to  10  percent. 

-  Contractors  using  the  Joint  Enterprise  Network  Manager 
(JENM)  were  able  to  plan,  configure,  and  load  MNVRs 
prior  to  the  MNVR  LUT.  Soldiers  must  perform  this  task 
in  combat  and  during  the  MNVR  IOT&E). 

-  Using  JENM,  Soldiers  could  not  monitor  or  manage 
MNVR  networks. 

-  The  MNVR  exceeded  its  reliability  requirements. 

-  MNVR  was  easy  to  use,  but  the  integration  of  the  radio 
into  tactical  vehicles  and  tactical  operation  centers  (TOCs) 
requires  improvement. 

-  MNVR  has  cybersecurity  vulnerabilities  that  could  degrade 
the  unit’s  ability  to  accomplish  its  mission. 

•  The  Army  is  developing  a  Test  and  Evaluation  Master  Plan 
(TEMP)  to  support  a  January  2016  Milestone  C  decision 
to  describe  post-Milestone  C  developmental  testing  and  an 
IOT&E. 

System 

•  The  Army’s  AN/VRC- 118  MNVR  program  evolved  from  the 
terminated  Joint  Tactical  Radio  System,  Ground  Mobile  Radio 
to  provide  software-programmable  digital  radios  to  support 
Army  tactical  communications  requirements  from  company 
through  brigade. 

•  The  Army  intends  the  MNVR  to: 

-  Operate  at  various  transmission  frequencies  using  the 
Soldier  Radio  Waveform  (SRW)  and  the  WNW. 


-  Bridge  the  upper  tactical  communications  networks  at 
brigade  and  battalion  with  the  lower  tactical  networks  at 
company  employing  a  terrestrial  radio  network. 

-  Provide  an  alternative  terrestrial  transmission  path  in  the 
absence  or  limited  availability  of  satellite  communications. 

•  The  MNVR  operates  up  to  75  watts  maximum  power  output 
for  WNW  and  up  to  5  watts  maximum  power  output  for  SRW. 

•  The  JENM  provides  the  means  to  plan,  load,  configure,  and 
monitor  MNVR  networks. 

•  The  MNVR  includes  both  vehicle-mounted  and  TOC  kit 
versions. 

•  The  MNVR  is  a  non-developmental  item  selected  through 
multi-vendor  competition. 

Mission 

•  Army  commanders  intend  to  use  the  MNVR  to: 

-  Provide  networked  communications  for  host  vehicles  and 
TOCs  during  all  aspects  of  military  operations 

-  Communicate  and  create  terrestrial  radio  networks  to 
exchange  voice,  video,  and  data  using  the  SRW  and  the 
WNW. 

-  Share  data  between  different  tactical  communication 
networks  and  mission  command  systems 

•  Signal  staffs  intend  to  use  the  JENM  to  plan,  load,  monitor, 
control,  and  report  on  network  operations  of  MNVR  networks 
running  SRW  and  WNW. 

Major  Contractor 

Harris  Corporation,  Tactical  Communications  -  Rochester,  New 

York 


MNVR  135 


F  Y  1  5  ARMY  PROGRAMS 


Activity 

•  In  November  2014,  the  Army  conducted  the  Government 
Integration  Test  (GIT)  at  the  Electronic  Proving  Ground 
in  Fort  Huachuca,  Arizona.  The  GIT  was  the  first  MNVR 
developmental  test  and  served  to  test  the  radio  against  its 
requirements.  The  purpose  of  GIT  was  to  provide  confidence 
that  the  MNVR  was  ready  to  proceed  to  the  planned  LUT. 
During  GIT,  MNVR: 

-  Demonstrated  WNW  data  and  SRW  voice  and  data 
requirements. 

-  Did  not  meet  message  completion  rate  requirements  over 
expected  transmission  distances  of  6  -  10  kilometers. 

-  Did  not  meet  reliability  requirements.  Due  to  a  large 
number  of  software  faults,  MNVR  demonstrated  less  than 
half  of  its  477  Mean  Time  Between  Essential  Function 
Failure  reliability  requirement. 

-  Did  not  interoperate  with  JENM  to  receive  radio 
configuration  files. 

-  Did  not  employ  WNW  Anti  Jam  or  SRW  Electronic 
Warfare  modes  during  test. 

•  In  January  and  February  2015,  the  Army  conducted  the 
Government  Regression  Test  (GRT)  at  the  Electronic 
Proving  Ground  in  Fort  Huachuca,  Arizona.  The  GRT  tested 
capabilities  that  were  not  met  or  were  not  tested  during  the 
GIT,  including  the  JENM,  which  was  not  available  during  the 
GIT.  During  GRT,  MNVR: 

-  Demonstrated  the  transfer  of  data  files  between  mission 
command  systems. 

-  Routed  SRW  voice  communications  over  multiple 
networks. 

-  Established  a  gateway  to  the  Warfighter  Information 
Network  -  Tactical  (WIN-T)  Net  Centric  Waveform 
satellite  network. 

-  Interoperated  with  JENM  to  load  MNVR  radio 
configuration  files. 

-  Did  not  employ  WNW  Anti  Jam  or  SRW  Electronic 
Warfare  modes  during  test. 

•  In  April  through  May  2015,  the  Army  conducted  a  MNVR 
LUT  as  part  of  the  NIE  15.2  at  Fort  Bliss,  Texas,  in 
accordance  with  a  DOT&E-approved  test  plan.  During  the 
LUT,  the  Army  employed  elements  of  the  2nd  Brigade,  1st 
Armored  Division  consisting  of  a  combined  arms  battalion,  a 
field  artillery  battalion,  and  a  brigade  headquarters  conducting 
missions  under  operationally  realistic  conditions.  Testing 
assessed  the  operational  employment  of  the  MNVR  providing 
a  terrestrial  communication  pathway  for  the  Joint  Battle 
Command  -  Platform  (JBC-P)  and  the  WIN-T  mission 
command  systems,  and  the  system’s  ability  to  establish  a 
mid-tier  network  to  link  the  lower  tactical  internet  with  the 
upper  tactical  internet. 

•  The  Army  is  developing  a  TEMP  to  support  a  January  2016 
Milestone  C  decision  to  describe  post-Milestone  C 
developmental  testing  and  an  IOT&E. 

Assessment 

•  DOT&E’s  evaluation  based  on  the  2015  MNVR  LUT  is: 


-  MNVR  enhanced  the  unit’s  mid-tier  network 
when  operating  within  a  full  network,  i.e.  satellite 
communications  were  available. 

-  In  a  reduced  satellite  network  environment,  the 
demonstrated  MNVR  WNW  network  message  completion 
rate  was  less  than  76  percent,  which  does  not  meet  the 
Army’s  MNVR  requirement  of  90  percent  at-the-halt  and 
85  percent  on-the-move. 

-  The  MNVR  WNW  network  experienced  faults  that 
prevented  4  of  12  battalion  MNVRs  from  sending  or 
receiving  any  data  for  extended  time  periods  (up  to 

36  hours).  These  MNVRs  were  within  line-of-sight  of 
other  MNVRs  and  should  have  had  communications. 
Existing  test  data  do  not  identify  whether  the  problem  was 
due  to  the  radio,  WNW,  or  network  configuration. 

-  The  unit  deployed  security  for  MNVR  retransmission 
vehicles  (necessary  to  provide  network  area  coverage). 

This  security  requirement  reduced  the  unit’s  combat  power 
by  up  to  10  percent. 

The  unit’s  employment  of  the  mid-tier  network  carried 
little  data  traffic  and  did  not  stress  the  bandwidth  capacity 
of  WNW. 

-  Contractors  using  the  JENM  were  able  to  plan,  configure, 
and  load  MNVRs  prior  to  the  MNVR  LUT.  Soldiers  must 
perform  this  task  in  combat  and  during  the  MNVR  IOT&E. 

-  Using  JENM,  Soldiers  could  not  monitor  or  manage 
MNVR  networks,  and  were  not  able  to  characterize  the 
health  of  individual  MNVR  nodes  or  individual  WNW 
links. 

-  The  MNVR  exceeded  its  reliability  requirements  within 
the  low  network  demands  of  NIE  15.2. 

-  The  MNVR  was  easy  to  use,  but  the  integration  of  the 
MNVR  into  tactical  vehicles  and  TOCs  did  not  support 
Soldier  ease  of  access  during  mission  operations. 

-  MNVR  has  cybersecurity  vulnerabilities  that  could  degrade 
the  unit’s  ability  to  accomplish  its  mission. 

•  The  Army  still  needs  to  conduct  a  complete  IOT&E  to  test 

all  features  of  MNVR  and  JENM  within  an  operationally 

representative  unit. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 

report  for  this  program. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Continue  development  of  the  MNVR  to  correct  the 
deficiencies  found  during  the  MNVR  LUT. 

2.  Develop  a  Milestone  C  TEMP  that  addresses  the 
developmental  and  operational  testing  that  will  support  a 
Full-Rate  Production  decision. 

3.  Plan  and  conduct  an  MNVR  IOT&E  using  an  Infantry 
Brigade  Combat  Team  equipped  with  JBC-P,  WIN-T 
Increment  2,  and  MNVR  in  accordance  with  an  approved 
MNVR  basis  of  issue  plan. 
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Mine  Resistant  Ambush  Protected  (MRAP)  MaxxPro 
Long  Wheel  Base  (LWB)  Ambulance 


Executive  Summary 

•  In  May  2015,  the  Army  Test  and  Evaluation  Command 
(ATEC)  conducted  the  Long  Wheel  Base  (LWB)  Ambulance 
Limited  User  Test  (LUT)  at  Yuma  Proving  Ground,  Arizona, 
in  accordance  with  the  DOT&E-approved  test  plan. 

•  The  LWB  Ambulance  is  operationally  effective  and 
operationally  suitable.  An  Army  unit  equipped  with  the  LWB 
Ambulance  can  provide  safe,  emergency  medical  care  and 
protected  transport  for  casualties  in  close  proximity  to  enemy 
forces. 

•  The  LWB  Ambulance  has  improved  capability  to  carry 
casualties  over  the  Dash  Ambulance: 

-  Accommodates  litter  patients  taller  than  5  feet  1 1  inches 

-  Holds  more  mission  essential  medical  equipment 

-  Provides  sufficient  space  for  medics  to  maneuver  within 
the  vehicle  to  treat  casualties 

•  The  LWB  Ambulance  is  reliable.  During  the  LWB  ambulance 
LUT,  the  vehicle  demonstrated  1,025  Mean  Miles  Between 
Operational  Mission  Failure  (MMBOMF)  versus  its 
operational  requirement  of  600  MMBOMF. 

•  The  LWB  ambidance  meets  its  required  levels  of  force 
protection,  and,  in  the  case  of  underbody  blast,  exceeds 
the  objective-level  requirements  at  some  locations.  This 
assessment  is  based  on  testing  conducted  with  the  LWB 
ambulance,  and  leverages  test  data  from  the  legacy  MaxxPro 
Mine  Resistant  Ambush  Protected  (MRAP)  and  the  MaxxPro 
Dash  with  MaxxPro  survivability  upgrade. 

System 

•  The  LWB  Ambulance  variant: 

-  Is  an  all-terrain  MRAP  ambidance  for  evaluating  and 
treating  casualties  from  forward  areas 

-  Has  a  rail,  trolley,  and  hoist  system  for  litter 
loading/unloading  and  gun  mounts  with  gunner 
protection  kits  on  which  to  mount  any  one  of  a  variety  of 
weapons,  such  as  the  M240B  medium  machine  gun,  the 
M2  .50  caliber  heavy  machine  gun,  and  the  Mk  19  grenade 
launcher 


MaxxPro  Long  Wheel 
Base  (LWB)  Ambulance 


-  Incorporates  current  Service  command  and  control  systems 
and  counter-IED  systems 

-  Incorporates  the  MaxxPro  Survivability  Upgrade  kit  that  is 
installed  on  MaxxPro  Dash  variants 

•  The  ambulance  is  operated  by  a  crew  of  three  medical  Service 
members  and  accommodates  two  patients  on  litters,  four 
ambulatory  patients,  or  a  combination  of  one  litter  patient  and 
two  ambulatory  patients. 

Mission 

Army  commanders  will  employ  units  equipped  with  the  LWB 
Ambulance  to  provide  safe,  emergency  medical  treatment  and 
protected  transport  for  casualties  in  close  proximity  to  enemy 
forces. 

Major  Contractor 

Navistar  Defense  -  Warrenville,  Illinois 


Activity 

•  As  part  of  the  Army  MRAP  enduring  fleet  requirement,  the 
program  is  producing  301  ambulances  to  be  placed  in  Army 
Pre -positioned  Stocks  and  training  base  in  June  2015.  The 
Army  procured  the  LWB  Ambulance  to  resolve  deficiencies 
with  the  small  interior  of  the  Dash  Ambulance  to  effectively 
care  and  safely  accommodate  litter  patients  taller  than  5  feet 
11  inches. 


•  ATEC  completed  developmental  testing  at  Aberdeen  Proving 
Ground,  Maryland,  and  Yuma  Proving  Ground,  Arizona,  in 
April  2015. 

•  In  May  2015,  ATEC  conducted  the  LWB  Ambulance  LUT 
at  Yuma  Proving  Ground,  Arizona,  in  accordance  with  the 
DOT&E-approved  test  plan. 


MRAP  LWB 
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Assessment 

•  The  LWB  Ambulance  is  operationally  effective  and 
operationally  suitable.  An  Army  unit  equipped  with  the 
LWB  Ambulance  can  provide  safe,  emergency  medical  care 
and  protected  transport  for  casualties  in  close  proximity  to 
enemy  forces.  During  the  LWB  Ambulance  LUT,  the  unit 
successfully  accomplished  four  out  of  five  medical  evacuation 
missions. 

•  The  LWB  Ambulance  has  improved  capability  to  carry 
casualties  over  the  Dash  Ambulance: 

-  Accommodates  litter  patients  taller  than  5  feet  1 1  inches 

-  Holds  more  mission  essential  medical  equipment 

-  Provides  sufficient  space  for  medics  to  maneuver  and  treat 
casualties  within  the  vehicle 

•  The  LWB  Ambulance  demonstrated  off-road  mobility  and 
maneuver  capability  similar  to  the  Dash  Ambulance. 

•  Although  medics  can  load  and  unload  litter  patients  with  the 
LWB  Ambulance  using  the  rail,  trolley,  and  hoist  system, 
loading  patients  is  hampered  due  to  misalignment  of  the  rails 
when  the  system  is  deployed. 

•  The  LWB  Ambulance  is  reliable.  During  the  LWB  Ambulance 
LUT,  the  vehicle  demonstrated  1,025  MMBOMF  versus  its 
operational  requirement  of  600  MMBOMF.  The  vehicle  can 
be  maintained  by  Soldiers  and  is  recoverable. 

•  The  LWB  Ambulance  cannot  safely  accommodate  litter 
patients  taller  than  6  feet  5  inches  due  to  the  location  of 
medical  and  vehicle  equipment  in  the  patient  compartment. 
The  equipment  is  in  close  proximity  to  the  patient’s  head  on 
the  litter  that  may  cause  additional  injury  to  the  litter  patient. 

•  The  height  of  the  gunner  stand/medic  seat  is  not  suitable  for 
shorter  Soldiers  to  effectively  provide  protective  fires.  During 


the  LUT,  shorter  Soldiers  positioned  the  gunner  stand/medic 
seat  in  an  unsafe  manner  to  raise  its  height  to  observe  their 
surroundings  and  engage  threats. 

•  The  LWB  Ambulance  lacks  stabilizing  handhold  and  inertial 
locking  seat  belts  to  allow  medics  to  safely  maneuver  within 
the  patient  compartment  and  treat  patients  during  transit. 
Medics  required  tactical  halts  to  treat  patients  during  the  LUT. 

•  LFT&E  conducted  in  FY14  indicates  the  vehicle  provides 
underbody  protection  beyond  its  required  levels.  The  LWB 
Ambulance  was  tested  against  underbody  mines  and  IEDs 
to  determine  potential  vulnerabilities  introduced  by  the 
integration  of  LWB  Ambulance  mission  equipment.  Test  data 
from  legacy  MaxxPro  MRAPs  establishing  compliance  with 
additional  force  protection  requirements  are  applicable  to 

the  LWB  Ambulance.  Of  the  MRAP  vehicles  the  DOD  has 
retained,  the  MaxxPro  MRAP  variants  set  the  standard  for 
underbody  blast  protection. 

Recommendations 

•  Status  of  Previous  Recommendations.  There  were  no  previous 
recommendations  for  this  variant. 

•  FY 15  Recommendations.  The  program  should: 

1 .  Relocate  the  installed  medical  and  vehicle  equipment 
with  the  objective  of  providing  additional  head  space  to 
accommodate  litter  patients  taller  than  6  feet  5  inches. 

2.  Improve  the  litter  rail  hoist  system  to  eliminate 
misalignment  of  the  rail  and  improve  patient  loading  time. 

3.  Redesign  gunner  stand/medic  seat  to  allow  height 
adjustment  to  accommodate  shorter  medics. 
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MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  Army  conducted  the  MQ-1C  Gray  Eagle  Unmanned 
Aircraft  System  (UAS)  FOT&E  at  Edwards  AFB,  California, 
and  the  National  Training  Center  (NTC),  Fort  Irwin, 

California,  May  14  through  June  12,  2015,  in  accordance 
with  the  DOT&E-approved  test  plan  and  Test  and  Evaluation 
Master  Plan. 

•  DOT&E  submitted  an  FOT&E  report  in  January  2016.  in  that 
report,  DOT&E  concludes: 

-  The  Gray  Eagle-equipped  unit  was  effective  at  conducting 
split-based  operations  while  operating  the  system  from 
two  separate  launch  and  recovery  sites  and  can  provide 
effective  reconnaissance,  surveillance,  and  security  support 
to  combat  units. 

-  Interoperability  with  the  One  System  Remote  Video 
Terminal  (OSRVT)  has  improved  since  the  2012  IOT&E. 

-  The  Gray  Eagle  system  is  operationally  suitable. 

-  Army  integration  of  Gray  Eagle  into  employment  concepts, 
and  development  of  tactics,  techniques,  and  procedures 
(TTP)  have  not  matured  since  IOT&E,  and  training  for  the 
FOT&E  unit  before  the  test  was  not  complete. 

System 

The  Gray  Eagle  UAS  is  composed  of  the  following  major 
components: 

•  Twelve  unmanned  aircraft,  each  with  a  common  sensor 
payload  with  an  electro-optical/infrared  (EO/IR)  and  a  Laser 
Range  Finder/Laser  Designator  capability,  a  STARLite 
Extended  Range  Synthetic  Aperture  Radar/Ground  Moving 
Target  Indicator  (SAR/GMTI)  radar,  and  an  Air  Data  Relay 
(ADR)  control  capability 

•  Each  aircraft  is  equipped  with  a  Standard  Equipment  Package 
that  includes  a  communications  relay  package,  Identification 
Friend-or-Foe  equipment,  and  Air  Traffic  Control  radios 

•  Each  aircraft  has  the  ability  to  carry  up  to  four  HELLFIRE  II 
P+  or  R  variant  missiles 

•  Six  Ground  Control  Stations  designated  as  the  Universal 
Ground  Control  Station  (UGCS) 


•  One  Mobile  Ground  Control  Station 

•  Seven  Tactical  Common  Datalinks  Ground  Data  Terminals 

•  Three  Satellite  Communications  Ground  Data  Terminals 

•  Twelve  Satellite  Communications  Air  Data  Terminals 

•  Six  Tactical  Automatic  Landing  Systems 

Mission 

Commanders  employ  Gray  Eagle  companies  to  conduct 
reconnaissance,  surveillance,  security,  attack,  and  command  and 
control  missions  that  support  assigned  division  combat  aviation 
brigade,  division  artillery,  battlefield  surveillance  brigade, 
Brigade  Combat  Teams,  and  other  Army  and  joint  force  units 
based  upon  the  division  commander’s  mission  priorities. 

Major  Contractor 

General  Atomics  Aeronautical  Systems,  Inc.,  Aircraft  Systems 
Group  -  Poway,  California 


Activity 

•  The  Army  conducted  the  Gray  Eagle  FOT&E  at  Edwards 
AFB,  California,  and  NTC,  Fort  Irwin,  California, 

May  14  through  June  12,  2015,  in  accordance  with  the 
DOT&E-approved  test  plan  and  Test  and  Evaluation  Master 
Plan. 

•  The  FOT&E  unit  conducted  missions  in  support  of  the 
Brigade  Combat  Team  conducting  a  training  rotation  at  the 
NTC.  This  combination  of  testing  with  training  created  a 


realistic,  challenging,  and  stressful  test  environment  for  the 
Gray  Eagle  Company.  The  company  flew  1,147  flight  hours 
during  test. 

•  The  Army  collected  data  from  the  FOT&E  to  assess  significant 
changes  within  the  company’s  organizational  structure  and 
system  components.  These  changes  include: 

-  Organizational  structure  from  one  flight  platoon  to 
three  identical  flight  platoons 
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-  The  capability  of  the  company  to  conduct  continuous  and 
simultaneous  split-based  operations  from  two  separate 
launch  and  recovery  sites 

-  The  replacement  of  the  One  System  Ground  Control 
Equipment  with  the  Universal  Ground  Control  Equipment 

-  The  replacement  of  the  three  portable  subsystems  with  one 
Mobile  Ground  Control  Station 

-  Upgrades  to  the  payloads  and  the  HELLFIRE  missile,  and 
significant  software  functionality  enhancements  made  to 
the  system  since  IOT&E 

•  During  the  FOT&E,  Gray  Eagle  crews  completed 

three  autonomous  F1ELLFIRE  missile  engagements  of  targets 
on  the  NTC  range  and  examined  the  capability  of  the  Gray 
Eagle  UAS  to  interface  with  OSRVT  Increment  II. 

•  DOT&E  submitted  a  combined  Gray  Eagle  FOT&E  and 
OSRVT  IOT&E  report  in  January  2016. 

Assessment 

•  During  FOT&E,  the  Gray  Eagle-equipped  unit  demonstrated 
it  was  effective  at  conducting  split-based  operations  from 
two  separate  launch  and  recovery  sites  and  can  provide 
effective  reconnaissance,  surveillance,  and  security  support 
to  combat  units.  Split-based  operations  testing  exposed 
single  points  of  failure  in  equipment  and  personnel  within  the 
Gray  Eagle  organizational  structure. 

•  The  Gray  Eagle  unit  contributed  to  the  situational  awareness 
of  supported  units  at  NTC.  In  surveys  of  subject  matter 
experts  with  the  supported  units,  32  of  36  respondents  agreed 
that  Gray  Eagle  provided  all  the  information  required.  Gray 
Eagle  crews  located  and  reported  enemy  vehicles  in  42  of 
48  mission  segments  when  at  least  one  threat  vehicle  was  in 
the  designated  search  area. 

•  The  Army  has  not  effectively  integrated  the  Gray  Eagle 
capabilities  into  combined  arms  combat  operations.  Gray 
Eagle  TTP  have  not  matured  since  the  2012  IOT&E.  Although 
not  fully  trained  before  the  test,  Gray  Eagle  Soldiers  became 
more  proficient  during  the  test,  but  many  remained  weak  on 
the  fundamentals  of  reconnaissance,  mission  planning,  and 
employment  of  Gray  Eagle  sensors.  Neither  the  Soldiers  in 
the  Gray  Eagle  unit  nor  those  requesting  Gray  Eagle  support 
understood  the  capabilities,  limitations,  and  employment  of  the 
SAR/GMTI  radar. 

•  Compared  to  the  OSRVT  Increment  I  performance  during  the 
2012  Gray  Eagle  IOT&E,  OSRVT  Increment  II  facilitated  an 
increased  level  of  situational  awareness  of  the  supported  unit 
by  providing  more  effective  full  motion  video.  Additional 
information  on  the  OSRVT  demonstrated  performance  may  be 
found  in  the  OSRVT  Increment  II  Annual  Report. 

•  Gray  Eagle  is  operationally  suitable.  The  Gray  Eagle  system 
demonstrated  an  operational  availability  of  87.4  percent, 
higher  than  the  76  percent  availability  demonstrated  during 
IOT&E.  High  availability,  low  Mean  Time  To  Repair,  and 
system  redundancy  allowed  operators  to  meet  operational 
requirements. 

•  The  Gray  Eagle  demonstrated  Key  System  Attribute  Mean 
Time  Between  System  Abort  of  23  hours  versus  the  42  hours 


requirement  for  the  Ground  Control  Equipment;  67  hours 
versus  63  hours  requirement  for  the  aircraft;  1,146  hours 
versus  300  requirement  for  the  common  sensor  payload;  and 
53  hours  versus  89  hours  requirement  for  the  SAR/GMTI 
radar. 

•  Integration  of  the  HELLFIRE  II  Romeo  missile  into  the 
Gray  Eagle  system  is  complete.  During  FOT&E,  the  unit 
successfully  demonstrated  the  ability  to  conduct  engagements 
with  the  HELLFIRE  II  Romeo  missile.  Three  autonomous 
engagements  of  targets  on  the  NTC  range  complex  were 
completed.  The  unit  hit  two  of  the  three  intended  targets.  The 
Army  demonstrated  HELLFIRE  engagement  via  the  ADR 
datalink  in  developmental  testing. 

•  The  Gray  Eagle  cybersecurity  posture  has  improved,  but  the 
system  remains  vulnerable  to  cyber  and  electronic  warfare 
threats. 

•  The  Gray  Eagle  Operator’s  Manual  states  that  flight  through 
light  rain  for  1  hour  falling  at  a  rate  of  up  to  'A  inch  per  hour 
poses  no  hazards  to  the  aircraft  and  that  operation  of  the 
aircraft  in  heavier  rain  than  this  rate  is  not  recommended. 
Visible  moisture  can  induce  mismatched  pressure  inputs  to  the 
engine  computer  resulting  in  a  warning  to  operators  to  land  as 
soon  as  possible,  creating  a  mission  abort.  During  FOT&E, 
the  unit  did  not  conduct  flight  operations  when  any  visible 
moisture  conditions  (fog,  clouds,  and  rain)  were  present.  This 
practice  limited  and  at  times  precluded  tactical  operations 
support. 

•  The  design  of  the  UGCS  shelter  is  an  improvement  over  the 
One  System  GCS  seen  in  previous  testing,  but  has  a  number 
of  deficiencies  that  reduce  operator  efficiency  and  increase 
operator  stress  and  fatigue.  Those  deficiencies  include: 

-  Operators  reported  that  headsets  became  uncomfortable 
over  a  period  of  time  and  pose  a  health  risk  because  the 
operators  must  share  the  few  headsets. 

-  The  Aviation  Mission  Planning  System  is  not  well 
integrated  into  the  UGCS  set  up  /starting  procedures. 
Operators  must  manually  input  most  pre-mission  data. 

-  Payload  and  Air  Vehicle  Operators  rated  the  usability  of 
UGCS  controls.  Operators  rated  the  controls  for  Link  16 
and  ADR  datalink  as  not  acceptable.  Payload  operators 
gave  marginal  ratings  to  the  controls  for  the  SAR/GMTI 
radar  and  to  the  interfaces  between  the  radar  and  EO/IR 
sensor. 

-  UGCS  employs  a  thumb-force  controller  on  the  keyboard 
that  replaced  the  mouse  track  ball  in  the  One  System 
GCS.  Operators  state  that  the  thumb-force  controller  is  not 
smooth  or  responsive  during  operation  and  after  periods 
of  non-use,  the  curser  tends  to  drift.  Both  the  Air  Vehicle 
and  Payload  operators  rated  the  thumb-force  controller 
usability  as  not  acceptable. 

•  Since  IOT&E,  the  Army  has  incorporated  improvements  to 
ADR  usability  and  has  reduced  the  number  of  steps  required 
to  establish  the  ADR  capability  from  130  to  93.  While  the 
28  percent  reduction  in  the  number  of  steps  is  admirable, 
further  improvements  need  to  be  made. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 

three  of  the  six  FY12  recommendations.  Outstanding  previous 

recommendations  include: 

1 .  Continue  to  develop  doctrine,  employment  concepts,  and 
TTP  to  fully  integrate  the  Gray  Eagle  unit  into  combat 
operations. 

2.  Train  operators  on  fundamentals  of  reconnaissance,  mission 
planning,  and  optimal  employment  of  the  Gray  Eagle. 

3.  Continue  to  simplify  procedures  for  operators  to  establish 
ADR  datalinks. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Implement  plans  to  modify  the  engine  computer  sensor 
inputs  that  will  allow  flight  in  visible  moisture. 

2.  Develop  TTP,  and  train  Gray  Eagle  operators  and 
supported  combat  units  in  the  utility  of  and  employment  of 
SAR/GMTI  radar  and  Link  16  capabilities. 


3.  Simplify  and  integrate  Aviation  Mission  Planning  System 
into  pre-mission  UGCS  setup  procedures. 

4.  Review  unit  organizational  documents  for  personnel  and 
equipment  single  points  of  failure  affecting  split-based 
operations  and  make  adjustments  to  those  documents  as 
necessary. 

5.  Eliminate  cybersecurity  vulnerabilities  and  confirm 
corrections  in  follow-on  testing. 

6.  Improve  UGCS  functionality  by  addressing  the  operator 
headset  and  thumb-force  controller  observations  made 
during  test. 

7.  Integrate  the  SAR/GMTI  radar  controls  and  displays  with 
the  EO/IR  controls  and  displays.  Target  detections  from  the 
MTI  radar  in  particular  should  overlay  the  same  map  that 
displays  the  location  of  the  air  vehicle  and  the  footprint  of 
the  EO/IR  sensor. 
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Nett  Warrior 


Executive  Summary 

•  The  Army  Test  and  Evaluation  Command  conducted  the  Nett 
Warrior  IOT&E  in  two  phases: 

-  A  mounted  calvary  troop  conducted  Phase  1  during 
Network  Integration  Evaluation  14.2  at  Fort  Bliss,  Texas 
(May  2014). 

-  A  dismounted  rifle  company  conducted  Phase  2  at 
Fort  Polk,  Louisiana  (November  2014). 

•  DOT&E  published  an  IOT&E  report  in  May  2015  to  support 
an  Army  Full-Rate  Production  (FRP)  Decision  Review  in 
June  2015.  Nett  Warrior  is  operationally  effective  in  mounted 
formations  at  the  platoon  and  troop  levels.  It  improved 
situational  awareness,  pre-mission  planning,  land  navigation, 
and  command  and  control. 

•  In  dismounted  infantry  units,  Nett  Warrior  is  effective  at 
the  platoon  level,  but  not  effective  at  the  company  level 
due  to  the  Manpack  radio’s  low  message  completion  rate  of 
position  location  information,  which  prevented  the  Company 
Commander  from  having  full  situational  awareness  of 
subordinate  units  during  operations. 

•  Nett  Warrior  is  operationally  suitable  and  reliable. 

•  The  Army  Acquisition  Executive  (AAE)  authorized  a 
fourth  low-rate  initial  production  to  procure  up  to  9,636  Nett 
Warrior  systems. 

System 

•  Nett  Warrior  is  a  dismounted  leader  situational  awareness 
system  for  use  during  combat  operations.  Nett  Warrior 
consists  of  the  following: 

-  End  User  Device,  a  commercial  off-the-shelf  Samsung 
smartphone 

-  Government-furnished  Rifleman  Radio  (AN/PRC- 154A) 
(Rifleman  Radio  is  a  separate  program  of  record,  with  a 
separate  article  later  in  this  Annual  Report.) 

-  Conformal  battery 

-  Connecting  cables 

-  Supporting  charging  equipment 

•  Periodic  Nett  Warrior  enhancements  integrate  improved 
commercial  smartphone  technologies. 

•  Nett  Warrior  graphically  displays  the  location  of  an  individual 
leader,  other  leaders,  friendly  vehicles,  battlefield  messages, 
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and  enemy  activity  on  a  digital  geo-referenced  map  image. 
Nett  Warrior  is  connected  through  a  secure  radio  to  the  Soldier 
Radio  Waveform  network  to  communicate  among  different 
echelons  using  voice,  data,  and  position  location  information 
messages. 

Mission 

•  Combatant  Commanders  employ  Nett  Warrior-equipped 
mounted  and  dismounted  leaders  as  part  of  a  Brigade  Combat 
Team  to  conduct  operations  (offensive,  defensive,  stability, 
and  defense  support  of  civil  authorities)  against  conventional 
or  unconventional  enemy  forces  in  all  types  of  terrain  and 
climate  conditions. 

•  Leaders  within  the  Brigade  Combat  Team  use  Nett  Warrior 
to  provide  improved  situational  awareness,  command  and 
control,  and  enhanced  communications. 

Major  Contractors 

AN/PRC- 154A  Rifleman  Radio: 

•  General  Dynamics  C4  Systems  -  Scottsdale,  Arizona 

•  Thales  Communications  Inc.  -  Clarksburg,  Maryland 


Activity 

•  The  Army  Test  and  Evaluation  Command  conducted  the  Nett 
Warrior  IOT&E  in  two  phases: 

-  Phase  1,  conducted  by  a  mounted  cavalry  troop  during 
Network  Integration  Evaluation  14.2  at  Fort  Bliss,  Texas, 
in  May  2014. 


-  Phase  2,  conducted  by  a  dismounted  rifle  company  at 
Fort  Polk,  Louisiana,  in  November  2014. 

•  DOT&E  published  an  IOT&E  report  in  May  2015  to  support 
the  FRP  Decision  Review  in  June  2015. 
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•  The  AAE  deferred  the  FRP  decision  pending  validation  of 
interoperability  of  Nett  Warrior  with  the  FRP  Manpack  radio. 
The  Army  and  DOT&E  will  validate  interoperability  during 
the  Manpack  radio’s  operational  test  in  FY17. 

•  The  AAE  authorized  a  fourth  low-rate  initial  production 
decision  to  procure  up  to  9,636  Nett  Warrior  systems  totaling 
20,503,  which  accounts  for  33  percent  of  the  Approved 
Acquisition  Objective. 

•  The  Army  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plans. 

Assessment 

•  Nett  Warrior  is  operationally  effective  in  mounted  formations 
at  the  platoon  and  troop  levels.  It  improved  situational 
awareness,  pre -mission  planning,  land  navigation,  and 
command  and  control. 

•  In  dismounted  infantry  units,  Nett  Warrior  is  effective  at 
the  platoon  level,  but  not  effective  at  the  company  level  due 
to  the  Manpack  radio’s  low  message  completion  rate  for 
position  location  information,  which  prevented  the  Company 
Commander  from  having  full  situational  awareness  of 
subordinate  units  during  operations. 


•  Nett  Warrior  is  operationally  suitable  and  reliable. 

•  Nett  Warrior  demonstrated  a  reliability  of  226  hours  Mean 
Time  Between  Essential  Function  Failure.  There  is  a 

90  percent  probability  of  operating  for  24  hours  without 
incurring  an  Essential  Function  Failure. 

•  Recharging  batteries  to  support  Nett  Warrior  is  an  increased 
logistical  burden.  The  process  to  operate  four  to  five 
generators  and  associated  battery  chargers  to  charge  all 
batteries  to  equip  a  light  infantry  company  takes  three  to 
four  Soldiers,  12  to  15  hours  a  day. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed  all 
previous  recommendations. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Improve  the  Soldier  Radio  Waveform  network  and 
associated  radios  to  increase  the  range  at  which  Soldiers 
and  leaders  can  use  Nett  Warrior. 

2.  Conduct  all  future  Rifleman  Radio  and  Manpack  radio 
operational  testing  with  the  Nett  Warrior  system. 
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One  System  Remote  Video  Terminal  (OSRVT) 

Increment  II 


Executive  Summary 

•  The  Army  conducted  the  One  System  Remote  Video  Terminal 
(OSRVT)  Increment  II  IOT&E  in  conjunction  with  the 
Gray  Eagle  Unmanned  Aircraft  System  (UAS)  FOT&E  at 
Edwards  AFB,  California,  and  the  National  Training  Center 
(NTC)  in  Fort  Irwin,  California,  from  May  14  through 

June  2,  2015.  The  Army  conducted  the  test  in  accordance  with 
the  DOT&E-approved  test  plan  and  the  Test  and  Evaluation 
Master  Plan. 

•  During  IOT&E,  the  OSRVT  demonstrated  the  capability 
to  interface  with  the  Gray  Eagle  UAS  and  support  combat 
operations. 

•  Compared  to  the  OSRVT  Increment  I  performance  during  the 
2012  Gray  Eagle  IOT&E,  OSRVT  Increment  II  facilitated  an 
increased  level  of  situational  awareness  of  the  supported  unit 
by  providing  more  effective  full  motion  video. 

•  The  Army  has  simplified  procedures  for  OSRVT  operators 
to  connect  to  and  receive  Gray  Eagle  video,  but  they  remain 
arduous  and  at  times  not  successful.  In  addition  to  receiving 
full  motion  video,  OSRVT  operators  exercised  control  of  the 
Gray  Eagle  electro  optical/infrared  (EO/IR)  sensor.  Operators 
achieved  this  capability,  known  as  Level  of  Interoperability  3 
(LOI  3),  on  six  missions  during  the  test,  but  had  to  frequently 
re-establish  the  LOI  3  datalink  during  each  of  those  missions. 

System 

•  The  OSRVT  system  is  a  portable  transceiver  unit  configured 
to  receive  imagery  and  metadata  from  selected  manned  and 
unmanned  aircraft.  It  is  modular  in  design  and  configured  for 
mounted  or  dismounted  operations  or  placement  in  a  Tactical 
Operations  Center. 

•  The  OSRVT  Increment  II  builds  upon  the  capabilities  of 
the  currently  fielded  OSRVT  Increment  I.  Increment  I  is  a 
remote  video,  radio  frequency-based,  line-of-sight  multi-band, 
receive-only  system.  In  addition  to  the  Increment  I 
capabilities,  the  OSRVT  Increment  II  provides  a  bi-directional 
communications  capability  that  allows  LOI  2  and  3  of 
unmanned  aircraft  platforms.  LOI  2  allows  direct  receipt  of 
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imagery  and/or  data  from  a  UAS.  LOI  3  allows  the  OSRVT 
operator  to  control  the  UAS  EO/IR  payload.  Increment  II 
also  provides  a  National  Security  Agency-approved  Type-1 
datalink  encryption  capability. 

•  The  OSRVT  has  three  major  subsystems: 

-  A  Multi-Band  Transceiver,  capable  of  receiving  and 
transmitting  digital  data 

-  Display/Computing  subsystem  (Tablet  Computer),  to 
receive,  view,  and  transmit  video  and  run  the  software  to 
provide  the  capability  to  overlay  telemetry  data  or  text 
onto  moving  maps  for  enhanced  geo-spatial  situational 
awareness 

-  Antennas,  including  the  Mobile  Directional  Antenna 
System  for  extended  range  beyond  50  kilometers 

Mission 

Commanders  and  Soldiers  at  all  echelons  employ  the  OSRVT 
system  to  gain  and  maintain  situational  awareness  during  an 
operation,  thus  enabling  the  user  to  visualize  the  battlefield  by 
receiving  full  motion  video  at  standoff  ranges. 

Major  Contractor 

Textron,  Unmanned  Systems  -  Hunt  Valley,  Maryland 


Activity 

•  The  Army  conducted  the  OSRVT  Increment  II  IOT&E  in 
conjunction  with  the  Gray  Eagle  UAS  FOT&E  at  Edwards 
AFB,  California,  and  the  NTC  in  Fort  Irwin,  California, 
from  May  14  through  June  12,  2015,  in  accordance  with 
the  DOT&E-approved  test  plan  and  the  Test  and  Evaluation 
Master  Plan. 


During  tactical  operations  in  the  IOT&E,  a  Stryker  Brigade 
Combat  Team  used  OSRVT  Increment  II  while  conducting 
a  training  rotation  at  the  NTC.  The  Brigade  Combat  Team 
employed  the  OSRVT  Increment  II  from  brigade  down  to 
company  level. 
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-  The  Army  collected  data  during  the  OSRVT  IOT&E  to 
assess  the  OSRVT-equipped  units’  increased  situational 
awareness  and  mission  accomplishment  as  a  result  of  the 
full  motion  video  and  LOI  3  capabilities  that  the  OSRVT 
Increment  II  provides. 

•  DOT&E  submitted  a  combined  Gray  Eagle  FOT&E  and 
OSRVT  IOT&E  report  in  January  2016. 

Assessment 

•  Compared  to  the  OSRVT  Increment  I  performance  during  the 
2012  Gray  Eagle  IOT&E,  OSRVT  Increment  II  facilitated  an 
increased  level  of  situational  awareness  of  the  supported  unit 
by  providing  more  effective  full  motion  video. 

•  The  Army  has  simplified  procedures  for  OSRVT  operators 
to  connect  to  and  receive  Gray  Eagle  video,  but  they  remain 
arduous  and  at  times  not  successful.  In  addition  to  receiving 
full  motion  video,  OSRVT  operators  exercised  control  of  the 
Gray  Eagle  EO/IR  sensor.  Operators  achieved  this  capability, 
known  as  LOI  3,  on  six  missions  during  the  test,  but  had  to 
frequently  re-establish  the  LOI  3  datalink  during  each  of  those 
missions. 

•  In  a  detailed  analysis  of  missions  supporting  units  at  NTC, 
units  receiving  OSRVT  video  from  Gray  Eagle  located  threat 
targets  in  a  shorter  timeframe  than  other  units  that  received 
Gray  Eagle  support,  but  without  OSRVT  video. 

•  Testing  demonstrated  that  LOI  3  is  capable  in  two  of  the 
three  Gray  Eagle  modes  of  control,  satellite  communication 
and  line-of-sight.  Testing  also  demonstrated  that  during 
the  Air  Data  Relay  mode  of  control,  the  LOI  3  capability  is 
not  possible.  This  technical  limitation  cannot  currently  be 
resolved. 

•  The  OSRVT  Increment  II  has  multiple  cables,  which  connect 
the  transceiver  to  the  other  system  components.  These  cable 
connections  on  the  transceiver  are  spaced  too  close  together 
and  connecting  them  is  difficult,  even  under  ideal  conditions. 


-  During  test,  when  under  limited-visibility  conditions  and 
while  adhering  to  noise  and  light  discipline,  Soldiers  were 
not  able  to  use  lights  and  therefore,  the  operators  could  not 
easily  connect  the  cables  to  the  transceiver.  This  caused 
the  OSRVT  operators  to  wait  for  daylight  to  complete 
emplacement  resulting  in  loss  of  time  that  the  system  could 
have  been  operational.  Operators  stated  that  the  multiple 
cables  should  be  color  coded  to  facilitate  ease  of  system  set 
up. 

•  The  OSRVT  is  vulnerable  to  cybersecurity  threats. 

•  The  OSRVT  Increment  II  Capability  Production  Document 
states  there  is  no  reliability  requirement  for  the  system.  The 
OSRVT  reliability  characterization  during  test  resulted  in 
zero  system  aborts  and  two  essential  function  failures  during 
the  total  892  LOI  2  (882.3  hours)  and  LOI  3  (9.7  hours) 
operating  time,  indicating  the  system  is  suitable  to  support 
combat  operations. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 
all  previous  recommendations  in  the  FY12  Gray  Eagle  Annual 
Report  concerning  OSRVT  operations.  However,  this  is  the 
first  annual  report  for  the  OSRVT  program. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Simplify  procedures  to  establish  and  improve  the  reliability 
of  LOI  3  connectivity. 

2.  Investigate  and  if  possible  rectify  the  inability  of  LOI  3 
functionality  when  the  Gray  Eagle  aircraft  is  flown  via  the 
Air  Data  Relay  mode  of  control. 

3.  Increase  the  space  between  the  cable  connections  on  the 
transceiver  to  provide  the  operators  more  room  to  connect 
the  required  system  cables  and  color  code  the  cables  to 
facilitate  ease  of  system  set  up. 

4.  Eliminate  cybersecurity  vulnerabilities  and  confirm 
corrections  in  follow-on  testing. 
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Patriot  Advanced  Capability-3  (PAC-3) 


Executive  Summary 

•  The  Army  conducted  Patriot  Advanced  Capability-3  (PAC-3) 
lethality  high-speed  sled  testing  in  May  2015  through 

June  2015,  and  testing  of  lethality  enhancements  from 
July  2015  through  October  2015. 

•  The  Army  began  the  Patriot  Post-Deployment  Build-8 
(PDB-8)  developmental  T&E  in  July  2015. 

System 

•  Patriot  is  a  mobile  air  and  missile  defense  system  that  counters 
missile  and  aircraft  threats. 

•  The  system  includes  the  following: 

-  C-band  multi-function  phased-array  radars  for  detecting, 
tracking,  classifying,  identifying,  and  discriminating 
targets  and  supporting  the  guidance  functions 

-  Battalion  and  battery  battle  management  elements 

-  Communications  Relay  Groups  and  Antenna  Mast  Groups 
for  communicating  between  battery  and  battalion  assets 

-  A  mix  of  Patriot  PAC-3  hit-to-kill  missiles  and  PAC-2  blast 
fragmentation  warhead  missiles  for  negating  missile  and 
aircraft  threats 

•  The  newest  version  of  the  PAC-3  missile  under  development 
is  the  PAC-3  Missile  Segment  Enhancement  (MSE).  The 
MSE  provides  increased  battlespace  defense  capabilities  and 
improved  lethality  over  prior  configuration  Patriot  missiles. 

•  Earlier  versions  of  Patriot  missiles  include  the  Patriot  Standard 
missile,  the  PAC-2  Anti-Tactical  Missile,  the  Guidance 
Enhanced  Missile  (GEM)  family  (includes  the  GEM-T  and 
GEM-C  missile  variants  intended  to  counter  tactical  ballistic 
missiles  and  cruise  missiles),  the  PAC-3  (baseline),  and  the 
PAC-3  Cost  Reduction  Initiative  variant. 

Mission 

Combatant  Commanders  use  the  Patriot  system  to  defend 
deployed  forces  and  critical  assets  from  missile  and  aircraft 


attack  and  to  defeat  enemy  surveillance  air  assets  (such  as 
unmanned  aerial  vehicles)  in  all  weather  conditions  and  in 
natural  and  induced  environments. 

Major  Contractors 

•  Prime:  Raytheon  Company,  Integrated  Defense 
Systems  -  Tewksbury,  Massachusetts  (ground  system  and 
PAC-2  and  prior  generation  missiles) 

•  PAC-3,  Cost  Reduction  Initiative,  and  MSE  Missiles: 
Lockheed  Martin  Corporation,  Missile  and  Fire 
Control  -  Grand  Prairie,  Texas 


Activity 

•  The  Army  conducted  high-speed  sled  testing  of  the  MSE 
missile  against  two  threat  sub-munition  warheads  at  the 
High-Speed  Sled  Test  Track  in  May  2015  through  June  2015, 
at  Holloman  Air  Force  Base,  New  Mexico. 

•  The  Army  began  the  PDB-8  developmental  T&E  in  July  2015, 
at  White  Sands  Missile  Range,  New  Mexico.  The  ground 
portion  of  this  testing  concluded  in  October  2015,  with 
developmental  flight  testing  scheduled  for  November  2015, 
December  2015,  March  2016,  and  July  2016. 

•  The  Army  conducted  testing  of  the  MSE  Lethality  Enhancer 
titanium  fragments  against  Composition  B  explosive  to 


validate  Jacobs-Roslund  equations  from  July  2015  through 
October  2015,  at  Army  Research  Laboratory,  Maryland. 
These  equations  predict  when  a  high-explosive  initiation 
should  occur  within  a  warhead  impacted  by  fragments. 

•  The  Army  plans  to  conduct  the  next  Patriot  operational  test, 
the  PDB-8  IOT&E,  beginning  in  September  2016.  This 
IOT&E  will  provide  information  to  support  the  Patriot 
Full-Rate  Production  decision  (including  the  MSE  missile). 

•  The  Army  conducted  all  testing  in  accordance  with  a 
DOT&E-approved  test  plan. 
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Assessment 

•  Problems  previously  discovered  during  the  PDB-7  Limited 
User  Test  (LUT),  if  not  corrected  during  PDB-8  development, 
could  adversely  affect  PDB-8  effectiveness,  suitability,  or 
survivability.  These  problems,  the  details  of  which  can  be 
found  in  the  classified  April  2013  Patriot  PDB-7  LUT  report, 
include: 

-  Patriot  performance  against  some  threats  improved 
compared  to  PDB-6.5,  but  there  were  degradations  in 
performance  against  other  threats.  Patriot  had  some 
effectiveness  shortfalls. 

-  Patriot  ground  system  reliability  did  not  meet  the  threshold 
requirement,  but  would  have  if  the  Patriot  radar  achieved 
its  allocated  reliability  goal. 

-  Patriot  ground  system  maintainability  did  not  meet  the 
threshold  requirement. 

-  Patriot  training  remained  inadequate  to  prepare  operators 
for  complex  Patriot  engagements.  This  was  true  during  the 
PDB  6.5  and  PDB-6  LUTs  as  well. 

-  Patriot  experienced  some  survivability  and  cybersecurity 
shortfalls. 

•  The  MSE  high-speed  sled  test  data  are  being  analyzed  to 
validate  lethality  models  of  MSE  lethality  against  the  tested 
targets. 

•  The  lethality  enhancer  contribution  to  MSE  lethality  against 
air-breathing  targets  is  to  be  determined.  The  requirement 
for  additional  sled  testing  is  contingent  upon  the  results  of 
simulations  of  air-breathing  target  engagements  throughout  the 
MSE  battlespace. 

•  Data  analysis  for  the  PDB-8  DTE  is  ongoing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  satisfactorily 
addressed  14  of  the  previous  23  recommendations.  The  Army 
should  continue  to  address  the  following  recommendations: 


1 .  Conduct  Patriot  air  and  missile  defense  testing  during 
joint  and  coalition  exercises  that  include  large  numbers 
of  different  aircraft  types,  sensors,  battle  management 
elements,  and  weapons  systems.  Conduct  Red  Team 
penetration  testing  during  joint  exercises  to  test  Patriot 
cybersecurity. 

2.  Conduct  a  Patriot  flight  test  against  an  anti-radiation  missile 
target  to  validate  models  and  simulations. 

3.  Improve  Patriot  training  to  ensure  Patriot  operators  are 
prepared  to  use  the  system  in  combat. 

4.  Elave  Patriot  participate  with  live  missiles  in  Terminal  High 
Altitude  Area  Defense  (THAAD)  flight  testing  to  determine 
Patriot-to-THAAD  interoperability  and  the  capability  for 
Patriot  to  intercept  tactical  ballistic  missile  targets  that  are 
not  intercepted  by  THAAD. 

5.  Collect  operational  reliability  data  on  Patriot  systems  in 
the  field  so  that  the  Mean  Time  Between  Critical  Mission 
Failure  can  be  calculated. 

6.  Use  test  units  for  future  Patriot  operational  tests  that 
have  operationally  representative  distributions  in  Soldier 
proficiency. 

7.  Conduct  future  operational  flight  tests  with  unannounced 
target  launches  within  extended  launch  windows. 

8.  Improve  Patriot  radar  reliability. 

9.  Obtain  the  data  required  to  validate  GEM  missile  blast 
lethality  in  the  Lethality  End  Game  Simulation. 

FY15  Recommendations.  None. 
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Precision  Guidance  Kit  (PGK) 


Executive  Summary 

•  In  May  2015,  the  Army  conducted  the  Precision  Guidance  Kit 
(PGK)  IOT&E  in  accordance  with  a  DOT&E-approved  test 
plan. 

•  PGK’s  demonstrated  accuracy  and  reliability  achieved  the 
Army’s  desired  mission  effects.  Remaining  reliability 
failure  modes  should  be  addressed  to  further  enhance  PGK 
effectiveness. 

•  In  January  2016,  DOT&E  published  an  IOT&E  report 
assessing  the  following: 

-  PGK  is  operationally  effective.  A  field  artillery  unit 
equipped  with  PGK  can  deliver  effective,  near-precision 
fires  when  firing  PGK-fuzed  conventional,  unguided 
155  mm  high-explosive  projectiles. 

-  PGK  is  accurate.  PGK  exceeded  its  accuracy  requirement 
of  50  meters  Circular  Error  Probable  (CEP)  by 
demonstrating  a  median  radial  miss  distance  of  10  meters 
in  accuracy  testing.  Accuracy  data  indicate  that  with 

90  percent  confidence,  the  true  CEP  is  less  than  or  equal  to 
20.9  meters. 

-  PGK  is  operationally  suitable.  PGK  met  its  reliability 
requirement  of  92.0  percent  at  a  point  estimate 

(92.1  percent)  but  not  with  confidence.  PGK’s  achieved 
accuracy  causes  PGK-fuzed  projectiles  to  exceed  expected 
effectiveness  even  with  a  reliability  that  does  not  meet  the 
reliability  requirement  with  confidence. 

-  PGK  is  survivable.  Cybersecurity  assessments  identified 
vulnerabilities  that  indicate  PGK  may  be  susceptible  to 
cyber  threats  with  physical  access  to  the  fuze. 

-  Cybersecurity  testing  of  PGK  and  the  M777A2 
lightweight,  towed  155  mm  howitzer  identified 
vulnerabilities  which  affect  the  operational  employment 
of  PGK.  Cybersecurity  assessments  of  the  Army’s 
end-to-end  artillery  fire  support  mission-processing  system 
should  be  conducted  to  identify  and  support  mitigation  of 
vulnerabilities,  which  could  affect  the  effectiveness  of  all 
artillery  fire  support  systems. 

-  A  Full-Rate  Production  (FRP)  decision  is  scheduled  for 
2QFY16. 


•  The  PGK  will  operate  with  existing  and  developmental 
artillery  systems  that  have  digital  fire  control  systems  and 
inductive  fuze  setters,  such  as  the  M777A2  Lightweight 
Towed  Howitzer,  the  M109A6  Paladin  Self-Propelled 
Howitzer,  and  the  M109A7  Paladin  Integrated  Management 
Self-Propelled  Howitzer. 

•  The  procurement  objective  is  102,921  PGK  fuzes.  The  Army 
plans  to  enter  full-rate  production  in  2QFY16. 

Mission 

Field  artillery  units  employ  PGK-fuzed  projectiles  to  support 
maneuver  units  with  indirect  fires  with  less  than  a  50-meter 
accuracy.  PGK-fuzed  projectile  accuracy  allows  field  artillery 
units  to  fire  fewer  projectiles  to  achieve  comparable  effects  of 
conventionally-fuzed  artillery  ammunition. 


System 

•  PGK  is  a  combined  fuze  and  GPS  guidance  module  that 
improves  the  ballistic  accuracy  of  the  current  stockpile  of 
high-explosive,  field  artillery  projectiles. 

•  The  Army  developed  PGK  for  155  mm,  high-explosive 
projectiles  (M795  and  M549A1)  with  a  threshold  accuracy  of 
50  meters  CEP  and  objective  accuracy  of  30  meters  CEP. 


Major  Contractor 

Orbital  ATK  Advanced  Weapons  Division  -  Plymouth, 
Minnesota 
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Activity 

•  Following  an  unsuccessful  attempt  to  move  the  beyond 
low-rate  initial  production  line  from  Minnesota  to 

West  Virginia  in  2013,  Orbital  ATK  moved  the  production  line 
back  to  the  original  contractor  production  facility  in  Minnesota 
in  2014. 

•  In  November  2014,  the  contractor  delivered  the  First  Article 
sample  of  PGK’s  from  the  Minnesota  production  line  to 
the  government.  These  PGK’s  successfully  completed  a 
First  Article  Acceptance  Test  and  safety  testing  at  Yuma 
Proving  Ground,  Arizona,  in  December  2014,  and  cold  regions 
testing  at  Fort  Greely,  Alaska,  in  January  2015. 

•  In  March  2015,  the  Army  conducted  a  cybersecurity 
vulnerability  and  penetration  assessment  of  the  PGK  fuze  and 
the  Army’s  M777A2  lightweight,  towed  155  mm  howitzer. 

•  DOT&E  approved  an  updated  PGK  Test  and  Evaluation 
Master  Plan  on  April  24,  2015. 

•  In  April  2015,  the  Army  conducted  the  first  of  four 

Lot  Acceptance  Tests  (LATs)  to  be  conducted  in  FY15  in 
support  of  the  scheduled  FRP  decision.  Following  successful 
completion  of  these  tests,  the  Army  accepted  1,539  low-rate 
initial  production  PGKs  for  fielding  to  Army  units. 

•  In  May  2015,  the  Army  conducted  the  PGK  IOT&E  in 
accordance  with  a  DOT&E-approved  test  plan. 

-  During  the  test,  a  towed  howitzer  battery  from  the  Army’s 
10th  Mountain  Division  fired  24  tactical  PGK  missions 
from  M777A2  howitzers  against  an  array  of  threat 
representative  targets. 

-  The  Army  conducted  an  Adversarial  Assessment  to 
determine  the  ability  of  a  unit  equipped  with  PGK 
to  protect,  defend,  recover  and  restore  effective  unit 
operations  while  withstanding  validated  and  representative 
cyber  threat  activity. 

•  In  January  2016,  DOT&E  published  an  IOT&E  report 
supporting  the  Army’s  planned  FRP  decision.  The  report 
analyzed  data  from  testing,  three  of  the  four  LATs,  and 
multiple  developmental  performance  and  safety  test  events 
conducted  prior  to  September  2015. 

Assessment 

•  DOT&E  assessed  the  following  based  on  observations  from 
the  IOT&E  and  LATs  1  through  3: 

-  PGK  is  operationally  effective.  A  field  artillery  unit 
equipped  with  PGK  can  provide  effective,  near-precision 


indirect  fires  when  firing  PGK-fuzed  conventional, 
unguided  155  mm  high-explosive  projectiles  in  support  of 
maneuver  units. 

-  PGK  is  accurate.  PGK  exceeded  its  accuracy  requirement 
of  50  meters  CEP  by  demonstrating  a  median  radial  miss 
distance  of  10  meters  in  accuracy  testing.  Accuracy  data 
indicate  that  with  90  percent  confidence  the  true  CEP  is 
less  than  or  equal  to  20.9  meters. 

-  PGK  is  operationally  suitable.  In  LATs  1  through  3,  PGK 
met  its  reliability  requirement  of  92.0  percent  at  a  point 
estimate  (92.1  percent)  but  not  with  confidence.  The  lower 
bound  of  an  80  percent  confidence  interval  is  88.7  percent. 
Correction  of  recurring  reliability  failure  modes  will  permit 
PGK  to  meet  its  reliability  requirement  with  confidence. 
PGK’s  achieved  accuracy  causes  PGK-fuzed  projectiles  to 
exceed  expected  effectiveness  even  with  a  reliability  that 
does  not  meet  the  reliability  requirement  with  confidence. 

-  PGK  is  survivable.  Cybersecurity  assessments  identified 
vulnerabilities  PGK  may  be  susceptible  to  cyber  threats. 
Cybersecurity  testing  showed  a  need  for  further  testing 
of  the  Army’s  artillery  fire  support  command  and  control 
systems. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 

previous  recommendations. 

•  FY 15  Recommendations.  The  Army  should: 

1 .  Continue  to  conduct  failure  mode  investigations  and  take 
corrective  actions  to  address  remaining  reliability  shortfalls 
and  meet  PGKs  reliability  requirement  with  confidence. 

2.  Perform  cooperative  and  adversarial  cybersecurity 
assessments  on  all  elements  of  the  artillery’s  end-to-end 
fire  support  mission  processing  system  and  take  necessary 
actions  to  mitigate  or  eliminate  these  vulnerabilities. 
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Q-53  Counterfire  Target  Acquisition  Radar  System 


Executive  Summary 

•  In  June  2015,  the  Army  conducted  the  Q-53  radar  IOT&E  2 
at  Yuma  Proving  Ground,  Arizona.  Soldier  crews  operated 
two  Q-53  radars  during  five,  continuous  72-hour  record 
test  scenarios  observing  mortar,  artillery,  and  rocket  fires. 
Soldiers  conducted  counterfire  operations  based  on  the  tactical 
scenario. 

•  The  Q-53  is  operationally  effective  for  single-fired  rocket, 
artillery,  and  mortar  munitions.  The  Q-53  is  not  operationally 
effective  for  volley-fired  mortar  munitions. 

•  The  radar  will  report  false  targets  when  no  projectiles  are 
in  the  search  area.  A  false  target  occurs  when  the  radar 
determines  that  a  threat  weapon  is  firing,  when  none  is  present. 

•  The  radar  is  required  to  characterize  projectiles  as  a  mortar, 
artillery,  or  rocket  fire.  The  radar  correctly  characterized 
every  single-fired  mortar  shot  as  a  mortar.  The  radar  often 
mischaracterizes  single-fired  rockets  and  artillery  as  mortars. 

•  The  Q-53  demonstrated  an  operational  availability  of 
0.99  during  IOT&E  2  (0.95  required)  indicating  the  radar  is 
operationally  suitable.  The  demonstrated  performance  of  the 
Q-53  during  IOT&E  2  indicates  it  is  not  meeting  reliability  or 
maintainability  requirements. 

•  The  Q-53  has  improved  cybersecurity  from  IOT&E  1  and  is 
survivable. 

•  In  October  2015,  DOT&E  submitted  an  IOT&E  report 
detailing  the  results  of  testing. 

•  The  Army  Program  Executive  Officer  for  Missile  and  Space 
will  make  a  Full-Rate  Production  decision  in  November  2015. 
The  Army  intends  to  procure  136  Q-53  Program  of  Record 
radars. 

System 

•  The  Q-53  Counterfire  Target  Acquisition  Radar  System  is  a 
mobile  radar  system  designed  to  detect,  classify,  and  track 
projectiles  fired  from  mortar,  artillery,  and  rocket  systems 
using  a  90-degree  or  continuous  360-degree  search  sector. 

•  The  Army  intends  the  radar  to  provide  target  location  for 
threat  indirect  fire  systems  with  sufficient  timeliness  and 
accuracy  for  effective  counterfire. 

•  The  Q-53  is  designed  to  operate  with  the  Counter  -  Rocket, 
Artillery,  Mortar  system  and  the  future  Indirect  Fire  Protection 
Capability  system. 


•  The  Army  intends  to  field  the  Q-53  radar  to  the  target 
acquisition  platoons  in  Brigade  Combat  Teams,  target 
acquisition  batteries  in  Field  Artillery  Brigades  and  Division 
Artillery  headquarters  to  replace  the  legacy  AN/TPQ-3  6  and 
AN/TPQ-37  Firefinder  radars. 

•  The  Q-53  is  operated  by  a  crew  of  five  Soldiers  and 
transportable  by  C-17  aircraft.  Two  Family  of  Medium 
Tactical  Vehicle  trucks  provide  battlefield  mobility. 

•  The  Army  contracted  with  Lockheed  Martin  Missile  Systems 
and  Sensors  to  develop  and  field  38  Quick  Reaction  Capability 
radars  to  support  an  Urgent  Material  Release.  The  Army 
intends  to  procure  136  Program  of  Record  Q-53  radars. 

Mission 

Field  Artillery  units  employ  the  Q-53  radar  to  protect  friendly 
forces  by  determining  the  accurate  location  of  threat  rocket, 
artillery,  and  mortar  systems  for  defeat  with  counterfire 
engagements.  Air  Defense  Artillery  units  integrate  the  Q-53 
radar  into  the  Counter  -  Rocket,  Artillery,  Mortar  and  Indirect 
Fire  Protection  Capability  System  to  warn  friendly  forces  and  to 
engage  incoming  threat  indirect  fires. 

Major  Contractor 

Lockheed  Martin  Missile  Systems  and  Training  -  Syracuse,  New 
York 


Activity 

•  Based  on  IOT&E  1  conducted  at  Yuma  Proving  Ground, 
Arizona,  in  May  2014,  the  Program  Executive  Officer 
for  Missiles  and  Space  decided  to  postpone  the  Full-Rate 
Production  decision  to  November  2015  and  conduct  a 


second  IOT&E  in  June  2015.  The  radar  did  not  meet  false 
target  rate  requirements,  reliability  requirements,  cyber 
vulnerabilities,  and  had  low  probability  of  detection  against 
volley-fired  munitions  during  IOT&E  1. 
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•  The  Army  completed  Developmental  Test  Phase  5  from 
January  through  February  2015.  Testing  focused  on  software 
changes  that  addressed  deficiencies  discovered  in  10T&E  1. 

•  The  Army  completed  a  developmental  capstone  event  in 
March  2015.  The  Program  Office  designed  the  capstone 
event  to  be  similar  to  IOT&E  2.  The  test  used  Soldier  crews 
operating  for  four,  72-hour  vignettes.  Units  deployed  the 
radars  in  the  90-degree  and  the  360-degree  modes. 

•  In  May  and  June  2015,  the  Army  conducted  the  Q-53 
IOT&E  2  at  Yuma  Proving  Ground,  Arizona,  in  accordance 
with  the  DOT&E-approved  Test  and  Evaluation  Master  Plan 
and  test  plan. 

-  Soldier  crews  operated  two  Q-53  radars  during  a  48-hour 
pilot  test  and  five,  72-hour  record  test  scenarios  observing 
mortar,  artillery,  and  rocket  fires. 

-  The  radars  operated  in  90-  and  360-degree  modes 
throughout  IOT&E  2. 

-  Army  electronic  warfare  and  cyber  teams  conducted 
attacks  against  the  test  unit  during  one  72-hour  period. 

•  DOT&E  submitted  the  Q-53  IOT&E  2  report  in  October  2015 
and  is  working  with  the  Army  to  develop  the  scope  and  details 
of  all  follow-on  testing. 

Assessment 

•  Based  on  IOT&E  2  results,  DOT&E  assessed  the  following: 

-  The  Q-53  is  operationally  effective  for  single-fired 
munitions  and  volley-fired  artillery.  The  radar  is  not 
effective  acquiring  volley-fired  mortars.  Volley-fire  is  a 
common  technique  used  by  a  variety  of  threat  nations  and 
an  important  component  of  an  operational  evaluation  for 
the  counterfire  radar. 

■  The  Q-53  consistently  and  accurately  detects  single-fired 
munitions  and  volley-fired  artillery. 

■  For  volley-fired  weapons,  the  Q-53  provided  consistent 
counterfire  acquisitions  for  artillery  projectiles  while 
operating  in  the  90-degree  modes. 

■  The  radar  had  problems  acquiring  volley-fired  mortars 
in  360-degree  and  90-degree  modes  and  volley-fired 
artillery  in  the  360-degree  mode.  Volley-fired  rockets 
were  not  tested.  The  radar  does  not  characterize  artillery 
and  rockets  properly.  After  acquiring  a  projectile,  the 
radar  is  required  to  characterize  the  projectile  as  mortar, 
artillery,  or  rocket.  The  radar  characterizes  mortars 
correctly,  but  often  mischaracterizes  rockets  and  artillery 
as  mortars.  Incorrect  characterizations  could  result  in 
ineffective  counterfire  missions.  The  Program  Office 

is  investigating  ways  to  improve  Q-53 ’s  ability  to 
characterize  projectiles. 

■  The  Q-53  met  false  target  rate  requirements  for  the 
360-  and  90-degree  normal  operating  modes,  but  not  for 
the  90-degree  short  range  optimized  mode.  While  in 
the  360-degree,  90-degree  normal,  and  90-degree  short 
range  optimized  modes,  the  radar  averaged,  0.5,  0.7, 
and  6.6  false  targets  per  12  radiating  hours,  respectively. 


The  Army  requires  the  Q-53  radar  to  have  no  more 
than  one  false  target  location  per  12  radiating  hours. 
Operators  are  not  able  to  distinguish  between  real  and 
false  targets,  which  can  result  in  wasted  counterfire 
missions  and  loss  of  confidence  in  the  radar.  When 
operating  near  an  air  station  in  IOT&E  2,  the  Q-53  had 
high  false  target  rates  while  in  the  90-degree  normal 
operating  mode.  These  rates  are  likely  due  to  activity  at 
the  air  station.  The  Program  Office  is  investigating  ways 
to  reduce  Q-53’s  false  target  rate. 

■  The  test  did  not  include  240  mm  rockets  or  122  mm 
cannon  artillery.  These  munitions  will  be  addressed  in 
FOT&E. 

■  In  the  90-degree  modes,  the  radar  incorrectly  uses  Digital 
Terrain  Elevation  Data  to  calculate  the  terrain  mask, 
causing  some  projectile  trajectories  to  travel  below  the 
radar  beams.  During  IOT&E  2,  18  of  188  threat  missions 
experienced  this  deficiency.  The  conditions  under  which 
this  deficiency  occurred  are  terrain  dependent  and  may 
occur  in  mountainous  terrain.  The  Program  Office 
discovered  this  problem  in  developmental  testing  prior  to 
IOT&E  2.  The  Program  Office  has  developed  a  fix  and 
are  testing  it. 

-  The  Q-53  is  operationally  suitable  primarily  due  to  its  high 
operational  availability.  During  IOT&E  2,  the  Q-53  radar 
was  available  for  496  of  500  hours  (99  percent).  This 
exceeded  the  95  percent  availability  requirement. 

■  The  demonstrated  performance  of  the  Q-53  radar  during 
the  IOT&E  2  indicates  that  the  program  is  not  meeting 
reliability  and  maintainability  requirements.  The  radar 
did  not  meet  the  reliability  requirement  because  of  the 
total  number  of  failures. 

■  The  four  hours  of  down  time  were  the  residt  of 
eight  system  aborts.  Although  the  radar  experienced 
more  system  aborts  than  allowed  by  the  requirements 
threshold,  the  downtime  for  most  aborts  was  small. 

The  majority  (5  of  8)  of  the  system  abort  failures  were 
software -related  and  five  of  the  aborts  required  less 
than  30  minutes  to  resolve.  The  Q-53  is  survivable 
and  demonstrated  significant  improvement  over  cyber 
vulnerability  from  the  IOT&E  1  in  May  2014. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed 
two  of  the  three  previous  recommendations.  However,  the 
Army  still  should  improve  the  radar’s  capability  of  detecting 
volley-fired  projectiles  in  both  90-  and  360-degree  modes. 

•  FY 15  Recommendation.  The  Army  should: 

1.  Conduct  an  FOT&E  to  address  122  mm  cannon  and 

240  mm  rocket  performance,  as  well  as  changes  to  improve 
false  target  rates,  false  target  rates  near  air  stations, 
volley-fire  detection,  characterization,  cybersecurity,  and 
generator  replacement. 
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Rifleman  Radio 


Executive  Summary 

•  The  Army  tested  the  AN/PRC- 1 54A  Rifleman  Radio  in 
support  of  company-level  operations  conducted  in  woodland 
and  urban  terrain  during  the  Nett  Warrior  IOT&E  at 

Fort  Polk,  Louisiana,  in  November  2014. 

•  The  Rifleman  Radio  is  the  primary  communications  device  for 
the  Nett  Warrior  and  was  critical  in  assessing  the  Nett  Warrior 
operationally  effective  in  platoon  and  company-level  mounted 
formations,  as  well  as  in  dismounted  platoon-level  formations. 

•  The  Rifleman  Radio,  when  integrated  into  Nett  Warrior, 
experienced  two  suitability  deficiencies:  a  high  occurrence  of 
batteries  overheating,  and  short  battery  life. 

•  The  Soldier  Radio  Waveform  (SRW)  on  the  Rifleman  Radio  is 
not  survivable  against  cyber  attacks.  Details  are  in  DOT&E’s 
classified  annex  to  the  Nett  Warrior  IOT&E  report  dated 
May  2015. 

•  The  Army  is  conducting  a  full  and  open  competition  of  the 
Rifleman  Radio  between  the  Thales  AN/PRC- 154B(V)1  and 
Harris  AN/PRC- 159(V)1  radios  in  FY16  with  a  projected 
delivery  of  4,000  radios  in  2QFY17. 

System 

•  The  Handheld,  Manpack,  and  Small  Form  Fit  program 
evolved  from  the  Joint  Tactical  Radio  System  program  and 
provides  software-programmable  digital  radios  to  support  the 
Army’s  tactical  communications  requirements.  The  two  radios 
that  comprise  the  Handheld,  Manpack,  and  Small  Form  Fit 
program  are  the  AN/PRC-154A  Rifleman  Radio  and  the 
AN/PRC-155  Manpack  radio. 

•  The  Army  is  replacing  the  General  Dynamics/Thales 
AN/PRC- 154A  with  either  or  both  the  Thales 
AN/PRC- 154B(V)1  and  the  Harris  AN/PRC- 159(V)1. 

•  The  Rifleman  Radio  is  a  handheld  network  radio  with  National 
Security  Agency  Type  1  encryption  for  Secret  communications 
and  data. 

•  The  Rifleman  Radio  is  capable  of  receiving/utilizing  external 
Precise  Position  Service  based  services. 

•  In  addition  to  functioning  as  a  stand-alone,  handheld  radio, 
the  Army  is  fielding  the  Rifleman  Radio  as  the  primary 
communication  device  for  the  Nett  Warrior  program.  The 
Army  is  developing  a  vehicle-mounting  kit  for  the  radio. 


Current  Low-Rate  Initial  Production  Rifleman  Radio  Full  and  Open  Competition  Rifleman  Radio 
PRC-1 54A  PRC-1 54B(V)1  &  PRC-1 59(V)1 

•  The  AN/PRC-154A  Rifleman  Radio  is  a  single-channel  radio 
with  a  commercial  GPS  receiver  to: 

-  Operate  at  various  transmission  frequencies  using  the 
SRW,  which  enables  the  radios  to  form  an  ad-hoc  data  and 
voice  communications  network  with  other  SRW-capable 
radios. 

-  Provide  5  watts  maximum  power  output  in  handheld,  and 
20  watts  maximum  power  output  in  vehicle  mount. 

-  Allow  Soldiers  to  transmit  position  location  information 
across  the  SRW  network.  The  position  location  capability 
of  the  Rifleman  Radio  is  disabled  when  connected  to 
Nett  Warrior. 

Mission 

Army  leaders  and  Soldiers  use  Rifleman  Radios  to  communicate 

and  create  networks  to  exchange  voice,  video,  and  data  using  the 

SRW  during  all  aspects  of  military  operations. 

Major  Contractors 

•  General  Dynamics,  C4  Systems  -  Scottsdale,  Arizona 
(AN/PRC-154A) 

•  Thales  Communications,  Inc.  -  Clarksburg,  Maryland 
(AN/PRC- 1 54 A  and  AN/PRC- 1 54B(  V)  1 ) 

•  Harris  Corporation  -  Rochester,  New  York 
(AN/PRC-159  (V)l) 


Activity 

•  The  Army  tested  the  AN/PRC- 1 54A  Rifleman  Radio  as  part 
of  the  Nett  Warrior  IOT&E  Phases  1  and  2  in  accordance  with 
DOT&E-approved  test  plans: 

-  Phase  I  occurred  in  May  2014  during  Network  Integration 
Evaluation  14.2  at  Fort  Bliss,  Texas. 


-  Phase  II  occurred  in  November  2014  at 
Fort  Polk,  Louisiana. 

•  The  Rifleman  Radio,  when  integrated  into  Nett  Warrior, 
experienced  two  suitability  deficiencies:  a  high  occurrence  of 
batteries  overheating,  and  short  battery  life. 


Rifleman  Radio 
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•  The  Army  is  conducting  a  hill  and  open  competition  of  the 
Rifleman  Radio  for  a  Full-Rate  Production  decision. 

•  The  Army  released  a  Request  for  Proposal  January  5,  2015. 

•  Harris  and  Thales  submitted  Rifleman  Radio  candidates  for 
Army  qualification  testing  in  July  2015. 

•  The  Army  has  planned  IOT &E  for  the  Thales 
AN/PRC- 1 54B(V)  1  and  the  Harris  AN/PRC-  159(V)1  for 
4QFY16. 

Assessment 

•  The  AN/PRC- 154A  Rifleman  Radio,  when  integrated  into  the 
Nett  Warrior  System,  experienced  two  suitability  deficiencies: 
a  high  occurrence  of  batteries  overheating,  and  short  battery 
life.  These  deficiencies  contributed  to  Soldiers  concluding 
that  the  radio  was  not  yet  acceptable  for  combat  in  its  current 
Nett  Warrior  configuration.  The  Nett  Warrior  Program 
Manager  made  changes  to  the  Nett  Warrior  system  to  mitigate 
Rifleman  Radio  suitability  deficiencies. 

•  At  the  Nett  Warrior  IOT&E  Phase  2  in  November  2014, 
the  AN/PRC- 154 A  Rifleman  Radio  supported  the  infantry 
company’s  rifle  platoons  as  they  operated  in  woodland  terrain 
with  both  voice  and  data  communications. 

•  The  SRW  on  the  Rifleman  Radio  is  not  survivable  against 
cyber  attacks.  Details  are  in  DOT&E's  classified  annex  to  the 
Nett  Warrior  IOT&E  report  dated  May  2015. 


•  The  radio  provided  voice  communications  across  the  platoon’s 
Area  of  Operations. 

•  The  Rifleman  Radio  demonstrated  improved  reliability 
compared  to  previous  test  residts,  demonstrating  a  Mean 
Time  Between  Essential  Function  Failure  of  328  hours,  with  a 
248  -  441  hour  80  percent  confidence  interval.  This  translates 
into  a  93  percent  chance  of  completing  a  24-hour  mission 
without  a  failure.  The  Rifleman  Radio  has  a  requirement  to 
complete  a  24-hour  mission  95  percent  of  the  time  without  an 
Essential  Function  Failure,  which  translates  into  a  Mean  Time 
Between  Essential  Function  Failure  of  477  hours. 

•  The  Thales  and  Harris  Rifleman  Radios  passed  qualification 
testing,  which  enables  them  to  proceed  to  a  product  manager’s 
customer  test  in  1QFY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Army  addressed  all 
previous  recommendations. 

•  FY15  Recommendation. 

1 .  The  Army  should  continue  testing  the  Rifleman  Radio  in  a 
configuration  that  is  operationally  representative  of  the  way 
it  will  be  employed  as  part  of  the  Nett  Warrior  system. 
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Warfighter  Information  Network  -  Tactical  (WIN-T) 


Executive  Summary 

•  In  May  2015,  the  Defense  Acquisition  Executive  (DAE) 
conducted  a  Warfighter  Information  Network  -  Tactical 
(WIN-T)  Increment  2  Full-Rate  Production  (FRP)  Defense 
Acquisition  Board  (DAB)  based  upon  the  October  through 
November  2014,  WIN-T  Increment  2  FOT&E  2.  The  DOT&E 
evaluation  was: 

-  The  Soldier  Network  Extension  (SNE)  with  Combat  Net 
Radio  (CNR)  Gateway  was  operationally  effective.  The 
Highband  Networking  Waveform  (HNW),  and  Tactical 
Relay  -  Tower  (TR-T)  were  not  operationally  effective. 

-  The  Mine  Resistant  Ambush  Protected  All-Terrain 
Vehicle  (M-AT  V)  SNE  and  Point  of  Presence  (PoP)  were 
operationally  suitable.  The  Stryker  SNE  and  PoP  were  not 
operationally  suitable. 

-  While  improved,  WIN-T  Increment  2  is  not  survivable  due 
to  cybersecurity  vulnerabilities. 

•  In  May  2015,  the  DAE  published  an  Acquisition  Decision 
Memorandum  (ADM)  that: 

-  Authorized  the  Army  to  enter  into  FRP  for  the  WIN-T 
Increment  2  program. 

-  Discontinued  quarterly  reporting  requirements  on  WIN-T 
Increment  2  reliability. 

-  Directed  the  preparation  of  corrective  action  plans  for 
HNW  and  Stryker  WIN-T  integration. 

-  Directed  an  independent  assessment  of  WIN-T 
cybersecurity  with  recommendations  for  improvement. 

•  In  September  2015,  the  DAE  delegated  the  Milestone 
Decision  Authority  for  WIN-T  Increment  2  to  the  Army. 

•  The  Army  did  not  complete  the  WIN-T  Increment  2  FRP 
Test  and  Evaluation  Master  Plan  (TEMP)  prior  to  the  FRP 
DAB.  To  complete  the  FRP  TEMP,  the  Army  continues  to 
design  an  FOT&E  to  test  WIN-T  Increment  2  configuration 
items  designed  to  support  light  brigades  with  downsized, 
air-transportable  WIN-T  assemblages. 

•  The  Army  continued  planning  for  an  FY 1 6  WIN-T  Increment 
3  FOT&E  to  assess  an  improved  version  of  the  Net  Centric 
Waveform  (NCW)  and  network  operations  tools. 

System 

•  The  Army  designed  the  WIN-T  as  a  three-tiered 
communications  architecture  (space,  terrestrial,  and  airborne) 
to  serve  as  the  Army’s  high-speed  and  high-capacity  tactical 
communications  network. 

•  The  Army  intends  WIN-T  to  provide  reliable,  secure,  and 
seamless  communications  for  units  operating  at  theater  level 
and  below. 

•  The  WIN-T  program  consists  of  three  funded  increments.  In 
May  2014,  the  DAE  approved  the  Army’s  request  to  stop 
development  of  the  Increment  3  aerial  tier  of  networked, 
airborne,  communications  relays  and  limit  Increment  3  to 


M-ATV  -  Mine  Resistant  Ambush  Protected 
(MRAP)  All-Terrain  Vehicle  (M-ATV) 

network  management  and  satellite  waveform  improvements. 

The  Army  intends  to  increase  procurement  of  WIN-T 

Increment  2  configuration  items  to  satisfy  the  number  of 

capability  sets  previously  planned  for  Increment  3. 

-  Increment  1:  “Networking  At-the-Halt”  enables  the 
exchange  of  voice,  video,  data,  and  imagery  throughout 
the  tactical  battlefield  using  a  Ku-  and  Ka-satellite-based 
network.  The  Army  has  fielded  WIN-T  Increment  1  to  its 
operational  forces. 

-  Increment  2:  “Initial  Networking  On-the-Move”  provides 
command  and  control  on-the-move  down  to  the  company 
level  for  maneuver  brigades  and  implements  an  improved 
network  security  architecture. 

-  WIN-T  Increment  2  supports  on-the-move  communications 
for  commanders  with  the  addition  of  the  PoP  and  the  SNE, 
and  provides  a  mobile  network  infrastructure  with  the 
Tactical  Communications  Node. 

-  WIN-T  Increment  2  provides  a  downsized, 
air-transportable  variant  of  High  Mobility  Multipurpose 
Wheeled  Vehicle  (HMMWV)  mounted  configuration  items 
to  support  the  Army’s  Global  Response  Force  and  other 
light  brigades. 

-  Increment  3:  “Full  Networking  On-the-Move”  was  to 
provide  full  mobility  command  and  control  for  all  Army 
field  commanders,  from  theater  to  company  level  using 
networked  airborne  communication  relays.  With  program 
reductions,  WIN-T  Increment  3  now  provides  enhanced 
network  management  and  an  improved  satellite  waveform 
to  WIN-T  Increments  1  and  2. 


M-ATV  Point  of  Presence 


M-ATV  Soldier  Network  Extension 


Stryker  Point  of  Presence 

1  -  Net-Centric  Waveform 

Antenna 

2  -  High-Band  Networking 

Waveform  Antenna 


Tactical  Comms  Node 


WIN-T 
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Mission 

Commanders  at  theater  level  and  below  will  use  WIN-T  to: 

•  Integrate  satellite-based  communications  capabilities  into 
an  everything-over-lnternet  Protocol  network  to  provide 
connectivity,  while  stationary,  across  an  extended,  non-linear 
battlefield  and  at  remote  locations  (Increment  1) 


•  Provide  division  and  below  maneuver  commanders  with 
mobile  communications  capabilities  to  support  initial 
command  and  control  on-the-move  (Increment  2) 

Major  Contractor 

General  Dynamics,  C4  Systems  -  Taunton,  Massachusetts 


Activity 

•  In  response  to  a  September  2013  DAB,  the  Army  conducted 
the  WIN-T  Increment  2  FOT&E  2  from  October  through 
November  2014,  as  part  of  Network  Integration  Evaluation 
(NIE)  15.1.  The  test  employed  the  2nd  Brigade,  1st  Armored 
Division  using  WIN-T  Increment  2  under  operationally 
realistic  conditions  at  Fort  Bliss,  Texas,  and  White  Sands 
Missile  Range,  New  Mexico.  The  WIN-T  Increment  2 
FOT&E  2  assessed  deficiencies  noted  during  FOT&E  1,  and 
assessed  the  integration  of  SNE  and  PoP  configuration  items 
into  Stryker  vehicles. 

•  In  May  2015,  DOT&E  published  a  WIN-T  Increment  2 
FOT&E  2  report  to  support  a  May  2015,  FRP  DAB. 

•  In  May  2015,  the  DAE  conducted  a  WIN-T  Increment  2  FRP 
DAB.  The  resulting  ADM: 

-  Authorized  the  Army  to  enter  into  FRP  for  the  WIN-T 
Increment  2  program. 

-  Authorized  closure  of  the  program’s  requirement  to  provide 
quarterly  progress  reports  on  reliability. 

-  Directed  the  Army  to  submit  by  June  30,  2015,  a  corrective 
action  plan  that  addresses  Stryker  integration  issues  and 
improvements  to  network  operations  tools  and  training 

to  optimize  performance  of  HNW,  TR-T,  and  Range 
Throughput  Extension  Kit  (RTEK).  The  Army  met  this 
submission  requirement. 

-  Directed  the  Army  to  provide  an  independent  cyber  design 
and  implementation  assessment  by  September  30,  2015, 
that  identifies  program  cybersecurity  vulnerabilities 

and  corrective  action  recommendations  for  future 
implementation.  The  Army  completed  this  requirement 
and  forwarded  the  results  to  the  DAE  on  October  30,  2015. 

•  The  Army  did  not  complete  the  WIN-T  Increment  2  FRP 
TEMP  prior  to  the  FRP  DAB.  To  complete  the  FRP  TEMP,  the 
Army  continues  to  design  an  FOT&E  to  test  WIN-T  Increment 
2  configuration  items  designed  to  support  light  brigades  with 
downsized,  air-transportable  WIN-T  assemblages. 

•  WIN-T  Increment  3  continued  to  design  an  FOT&E  for 
NIE  16.2  to  test  an  improved  NCW  and  enhanced  network 
operations  tools  that  will  be  fielded  to  WIN-T  Increments  1 
and  2  units. 

•  In  September  2015,  the  DAE  delegated  the  Milestone  Decision 
Authority  for  WIN-T  Increment  2  to  the  Army,  designating  the 
program  Acquisition  Category  1C. 


Assessment 

•  In  the  May  2015  WIN-T  Increment  2  FOT&E  2  report, 
DOT&E  assessed  the  following: 

-  The  SNE  with  CNR  Gateway  was  operationally  effective, 
providing  a  means  to  bridge  dispersed  radio  networks  and 
link  mission  command  applications  across  the  brigade  area 
of  operations. 

-  The  PoP,  SNE,  and  CNR  Gateway  provided  improved 
performance  for  Soldiers  which  included: 

■  An  improved  user  interface  that  allows  an  intuitive, 
easy-to-use  method  for  operations  and  troubleshooting. 

■  Improved  support  for  chat  and  voice  communications. 

-  The  Net  Centric  Waveform  was  operationally  effective, 
providing  a  stable  satellite  network  to  support  WIN-T 
communications. 

-  The  HNW  and  TR-T  were  not  effective  due  to  limited 
transmission  range  and  throughput  for  on-the-move  links, 
poor  quality  at-the-halt  links,  inability  to  maintain  a 
non-fragmented  network  in  the  absence  of  satellite,  lack  of 
use  of  the  RTEK,  and  poor  network  operations  tools. 

-  The  M-ATV  PoP  and  SNE  were  operationally  suitable  and 
met  their  reliability  requirements. 

-  The  Stryker  PoP  and  SNE  were  not  operationally  suitable 
due  to  poor  integration  of  WIN-T  Increment  2  equipment 
that  interfered  with  the  Soldiers’  performance  of  mission  to 
include: 

■  Displays  in  front  of  the  gunner’s  position 

■  Antennas  that  prevent  360-degree  gun  coverage 

■  Operations  with  the  engine  off  that  drains  batteries  to  the 
point  of  replacement 

■  Operating  with  the  Stryker  running  to  support  WIN-T 
does  not  allow  “Silent  Watch”  operations 

-  The  Stryker  PoP  met  its  reliability  requirement,  while  the 
Stryker  SNE  did  not. 

-  WIN-T  Increment  2  is  not  survivable.  Although  improved, 
WIN-T  Increment  2  continues  to  demonstrate  cybersecurity 
vulnerabilities.  This  is  a  complex  challenge  for  the 
Army  since  WIN-T  is  dependent  upon  the  cyber  defense 
capabilities  of  all  mission  command  systems  connected  to 
the  network. 

-  Improved  reliability  that  met  reliability  requirements 
for  the  M-ATV  PoP  and  SNE,  and  the  Stryker  PoP.  The 
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Stryker  SNE  did  not  meet  its  reliability  requirement,  but 
was  borderline. 

•  WIN-T  Increment  2  provides  HMMWV  mounted 
configuration  items  to  support  the  Army’s  Global  Response 
Force  and  other  light  brigades.  The  Army’s  WIN-T  Increment 
2  FOT&E  planning  for  this  configuration  set  has  focused  on 
the  Tactical  Communications  Node  -  Light  (TCN-L)  and 
Network  Operations  and  Security  Center  -  Light  (NOSC-L), 
but  has  not  planned  testing  for  the  HMMWV  mounted  PoP  and 
SNE  (as  equipped  in  light  brigades).  The  HMMWV-mounted 
PoP  and  SNE  have  not  been  assessed  in  an  operational  test. 
This  version  of  WIN-T  Increment  2  is  a  significant  redesign  of 
the  fielded  M-ATV  variants  and  requires  an  FOT&E  to  assess 
this  version’s  effectiveness,  suitability,  and  survivability. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  addressed 
one  of  four  previous  recommendations.  They  still  need  to 
conduct  the  planned  NCW  and  network  operations  FOT&E 


in  FY16,  plan  operational  testing  of  future  integration  of 

WIN-T  Increment  2  into  combat  vehicles,  and  improve  the 

transmission  range  of  HNW  and  employment  of  the  TR-T. 

•  FY 15  Recommendations.  The  Army  should: 

1.  Improve  WIN-T  Increment  2  cybersecurity  and  assess  its 
survivability  in  a  future  operational  test. 

2.  Improve  the  employment  of  the  HNW  with  TR-T  and 
RTEK  by  providing  tactics,  techniques,  and  procedures  with 
improved  training  and  enhanced  network  operations  tools. 

3.  Complete  the  WIN-T  Increment  2  FRP  TEMP. 

4.  Conduct  an  operational  test  to  assess  WIN-T  Increment  2 
HMMWV-mounted  configuration  items  designed  to  support 
the  Global  Response  Force  and  other  light  brigades. 

5.  Conduct  an  operational  test  to  assess  the  improved 
NCW  and  network  operations  tools  provided  by  WIN-T 
Increment  3. 

6.  Improve  Stryker  WIN-T  integration  and  demonstrate  these 
improvements  in  a  future  operational  test. 
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Acoustic  Rapid  Commercial  Off-the-Shelf  Insertion 
(A-RCI)  for  AN/BQQ-1 0(V)  Sonar 


Executive  Summary 

•  DOT&E  submitted  a  classified  FOT&E  report  on  the 
Advanced  Processing  Build  2011  (APB-11)  variant  of  the 
AN/BQQ-10(V)  Acoustic  Rapid  Commercial  Off-the-Shelf 
Insertion  (A-RCI)  sonar  system  in  November  2015. 

•  From  May  2013  through  August  2014,  the  Navy  completed 
operational  testing  on  the  APB- 11  variant  of  the  A-RCI  sonar 
system. 

-  Operational  test  phases  conducted  in  FY14  consisted  of 
Anti-Submarine  Warfare  (ASW)  against  a  diesel  electric 
submarine  (SSK)  target,  cybersecurity,  and  situational 
awareness  in  a  High  Density  Contact  Management 
(HDCM)  region  with  a  Light  Weight  Wide  Aperture  Array 
(LWWAA). 

-  Previously  conducted  test  phases  for  this  software  variant 
include  ASW  against  a  nuclear  submarine  (SSN)  target  and 
situational  awareness  in  an  HDCM  region  with  an  Active 
Low  Cost  Conformal  Array  (ALCCA). 

-  DOT&E  assessed  that  overall  mission  performance 
was  unchanged  from  previous  variants  of  the  system; 
however,  APB- 11  demonstrated  improvements  in  ASW 
and  situational  awareness  in  HDCM  missions  with  an 
LWWAA  over  previous  APB  variants. 

-  DOT&E  assessed  APB-11  demonstrated  improved 
reliability  and  was  suitable. 

-  All  test  phases  were  adequately  conducted  to  assess 
system  performance.  However,  there  were  a  number  of 
test  limitations,  which  precluded  a  full  assessment  of 
capabilities.  More  prominent  test  limitations  were: 

-  A  damaged  TB-29  array  limited  the  assessment  of  new 
single  leg  ranging  capabilities  and  the  use  the  new  Range 
Azimuth  (RAZ)  and  Range  Triage  Display  (RTD). 

-  The  RAZ/RTD  suffered  a  material  casualty  that  precluded 
assessing  their  mission  effect  on  situational  awareness  in 
an  HDCM  region. 

•  The  Navy  began  to  develop  its  operational  test  strategy 
and  associated  documentation  to  assess  the  upcoming 
APB- 1 3  variant  of  the  system.  The  Navy  intends  to  conduct 
operational  testing  of  APB- 13  in  late  FY16. 
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System 

The  A-RCI  sonar  system: 

•  Is  intended  to  maintain  an  advantage  in  acoustic  detecting 
threat  submarines. 

•  Processes  data  from  the  submarine’s  acoustic  arrays  (i.e., 
spherical  array,  large  aperture  bow  array,  hidl  array,  wide 
aperture  array,  conformal  array,  and  high-frequency  array) 
along  with  the  submarine’s  two  towed  arrays  (i.e.,  the  fat  line 
array  consisting  of  the  TB-16  or  TB-34,  and  the  thin  line  array 
consisting  of  the  TB-23  or  TB-29). 

Mission 

The  Operational  Commander  will  employ  submarines  equipped 

with  the  A-RCI  system  to: 

•  Search,  detect,  and  track  submarine  and  surface  vessels  in 
open-ocean  and  littoral  sea  environments  without  being 
counter-detected 

•  Search,  detect,  and  avoid  mines  and  other  submerged  objects 

•  Covertly  conduct  Intelligence,  Surveillance,  and 
Reconnaissance 

•  Covertly  conduct  Naval  Special  Warfare  missions 

•  Perform  under-ice  operations 

Major  Contractor 

A-RCI:  Lockheed  Martin  Maritime  Systems  and 

Sensors  -  Washington,  District  of  Columbia 


Activity 

•  From  May  2013  through  August  2014,  the  Navy  completed 
operational  testing  on  the  APB- 11  variant  of  the  A-RCI  sonar 
system. 

-  Operational  test  phases  conducted  in  FY14  consisted  of 
ASW  against  an  SSK  target,  cybersecurity,  and  situational 
awareness  in  an  HDCM  region  with  an  LWWAA. 


-  Previously  conducted  test  phases  for  this  software  variant 
included  ASW  against  an  SSN  target  and  situational 
awareness  in  an  HDCM  region  with  an  ALCCA. 

•  In  July  2014,  the  AN/BQQ-10(V)  A-RCI  sonar  system  was 
removed  from  DOT&E  oversight  due  to  resource  constraints. 
It  was  restored  to  oversight  in  October  2014  because  of 
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concern  with  the  system’s  performance  in  support  of  both 
the  Virginia  and,  eventually,  Ohio  Replacement  submarine 
programs.  DOT&E’s  resource  constraint  was  resolved  by 
eliminating  other  programs  that  were  not  as  critical  to  Virginia 
and  Ohio  Replacement  performance. 

•  In  September  2014,  DOT&E  submitted  an  interim 
memorandum  documenting  the  results  ofA-RCI  APB-11 
operational  testing  completed  and  analyzed  prior  to  removal 
from  oversight. 

•  In  November  2015,  DOT&E  submitted  a  classified  FOT&E 
report  on  the  APB- 1 1  variant  of  the  A-RCI  sonar  system, 
which  detailed  testing  phases  performed  and  analyzed  while 
temporarily  off  of  DOT&E  oversight. 

•  ASW  testing  and  situational  awareness  testing  with  an  ALCOA 
were  conducted  in  accordance  with  a  DOT&E-approved 

test  plan.  Situational  awareness  testing  in  HDCM  with  an 
LWWAA  and  cybersecurity  testing  were  not  conducted  with 
a  DOT&E-approved  test  plan  due  to  the  program  being 
temporarily  off  DOT&E  oversight. 

•  In  October  2014,  the  Navy  began  test  planning  for  the  APB-13 
variant  of  the  system,  which  is  expected  to  occur  in  late  FY16. 
Expected  test  events  include: 

-  At-sea  ASW  performance  assessment  against  an  SSN  or 
SSK  target 

-  In-lab  ASW  performance  assessment  against  various  threat 
targets 

-  At-sea  situational  awareness  in  an  HDCM  region  with  an 
ALCCA  and  LWWAA 

-  Cybersecurity 

Assessment 

•  DOT&E  determined  that  the  APB- 1 1  variant  of  the  A-RCI 
sonar  system’s  overall  mission  performance  remains 
unchanged  from  previous  assessments  and  further  observed 
an  improvement  in  system  reliability.  The  recently  released 
classified  DOT&E  FOT&E  report,  in  conjunction  with  the 
classified  interim  assessment  memorandum  dated  September 
10,  2014,  concluded  the  following  regarding  performance: 

-  For  ASW,  APB- 1 1  A-RCI  passive  sonar  capability  is 
effective  against  older  classes  of  submarines  in  some 
environments,  but  is  not  effective  in  all  environments  or 
against  modem  threats.  Despite  an  unchanged  overall 
assessment,  APB- 11  demonstrated  improved  operator 
performance  metrics  over  previous  APB  variants. 

-  The  APB- 1 1  A-RCI  sonar  system  is  not  effective  in 
supporting  operator  situational  awareness  and  contact 
management  in  areas  of  high-contact  density;  however, 
platforms  equipped  with  an  LWWA  demonstrated  improved 
performance  over  previous  APB  variants. 

-  APB-1 1  cybersecurity  is  not  effective  and  remains 
unchanged  from  previous  variants. 

-  The  APB- 1 1  A-RCI  sonar  system  is  operationally  suitable. 

•  Although  the  APB- 1 1  assessment  was  able  to  determine 
system  effectiveness  and  suitability,  there  were  several  test 
limitations.  Some  of  the  major  limitations  were: 


-  A  damaged  TB-29  array  that  limited  the  assessment  of 
new  single  leg  ranging  capabilities  and  the  use  of  the  new 
RAZ/RTD. 

-  The  RAZ/RTD  suffered  a  material  casualty  that  precluded 
an  assessment  of  its  impact  on  situational  awareness  in  an 
HDCM  region. 

•  Due  to  the  biennial  software  and  hardware  development  cycle, 
the  Navy  generates  and  approves  the  requirements  documents 
and  Test  and  Evaluation  Master  Plans  (TEMPs)  in  parallel 
with  APB  development  and  installation.  As  a  result,  the  fleet 
assumes  additional  risk,  since  most  operational  testing  is  not 
completed  before  the  system  is  initially  deployed. 

•  The  Navy’s  schedule-driven  process  prevents  operational  test 
results  from  directly  supporting  development  of  the  follow-on 
APBs.  For  example,  the  Navy  completed  operational  testing 
of  the  A-RCI  APB-09  sonar  system  in  early  FY12.  Due  to  the 
combination  of  the  late  completion  of  testing  and  the  Navy’s 
practice  of  issuing  an  updated  version  every  2  years,  data  from 
the  test  could  not  be  included  in  the  development  of  APB-11. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  made 
progress  in  addressing  22  of  the  37  previous  recommendations 
outlined  in  DOT&E’s  classified  FOT&E  report  on 

APB-09  dated  November  2012.  Of  the  15  remaining 
outstanding  recommendations,  the  significant  unclassified 
recommendations  are: 

1 .  Conduct  additional  testing  in  shallow  water  to  examine  the 
ship’s  ASW  capabilities  in  those  conditions. 

2.  Re-evaluate  the  use  of  the  current  time  difference  between 
system  and  operator  detection  times  as  the  ASW  Key 
Performance  Parameter  for  a  more  mission-oriented  metric 
to  accurately  categorize  system  effectiveness. 

3.  Evaluate  the  covertness  of  the  high-frequency  sonar  during 
a  future  submarine-on-submarine  test. 

4.  Determine  the  performance  of  the  A-RCI  sonar  system  in 
detecting  near  surface  mines. 

-  The  following  recommendations  from  the  FY12  Annual 
Report  remain  open.  In  the  upcoming  fiscal  year,  the  Navy 
should: 

1.  Consolidate  the  A-RCI  and  AN/BYG-1  TEMPs  and  test 
plans  into  an  Undersea  Enterprise  Capstone  document  to 
permit  efficiencies  in  testing. 

2.  Evaluate  A-RCI  metrics  to  improve  performance  under 
varying  environmental  conditions  and  to  focus  on  earlier 
and  longer  range  operator  detections. 

•  FY 15  Recommendations.  DOT&E’s  APB-1 1  FOT&E  report 
dated  November  2015  detailed  five  new  recommendations. 

The  Navy  needs  to  address  the  following  significant 
unclassified  recommendations: 

1.  Carry  forward  all  APB-11  test  objectives  not  evaluated 
including  the  ALCCA,  RAZ/RTD  user  interface,  TB-34 
triangulation  ranging  improvements,  and  TB-29  ranging 
algorithm  to  the  APB- 13  TEMP. 
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2.  Perform  an  ASW  event  against  a  high-end  SSK  at  least  with 
every  other  APB  variant  and  upon  introduction  of  new  wet 
end  sensor  or  software  capabilities  improving  ASW  mission 
capability. 

3.  Conduct  future  HDCM  situational  awareness  testing 
in  areas  that  provide  full  radar  coverage  to  support 
comparative  analysis  of  data. 
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Aegis  Modernization  Program 


Executive  Summary 

•  The  Navy  is  modernizing  the  Aegis  Weapon  System  (AWS) 
installed  on  Baseline  3  USS  Ticonderoga  (CG  47)  class 
cruisers  and  the  Flight  I  USS  Arleigh  Burke  (DDG  51) 
destroyers  to  the  AWS  Advanced  Capability  Build  2012 
(Baseline  9A  and  9C,  respectively).  New  construction  DDGs, 
beginning  with  USS  John  Finn  (DDG  113),  will  be  equipped 
with  Baseline  9C  as  well. 

•  Testing  completed  to  date  is  insufficient  to  make  a 
determination  of  operational  effectiveness  or  suitability  for 
Aegis  Baseline  9A  or  9C. 

•  In  accordance  with  National  Defense  Authorization  Act  of 
2008,  Section  231,  DOT&E  submitted  Early  Fielding  Reports 
in  July  and  November  2015  for  each  baseline  incident  in 
response  to  the  Navy’s  deployment  of  USS  Normandy  and 
USS  B enfold,  and  prior  to  the  completion  of  operational 
testing.  Testing  on  Baseline  9A  and  9C  ships  to  date  suggest 
that  area  air  defense  performance  against  subsonic  and 
supersonic  high-diving  targets  is  consistent  with  historical 
performance  against  comparable  threats;  however,  during 
operational  testing,  the  Navy  has  not  yet  demonstrated 
performance  against  more  stressing  presentations. 

•  In  February  2015,  the  Navy  commenced  Baseline  9A 
operational  testing  on  USS  Chancellorsville  (CG  62). 

One  planned  live  fire  event  was  deferred  due  to  target 
availability,  and  two  of  four  additional  planned  at-sea  events 
were  not  completed  because  of  test  execution  problems. 

These  unexecuted  operational  test  events  are  currently 
scheduled  for  late  1QFY16. 

•  From  November  2014  through  April  2015,  as  part  of 
Combat  System  Ship  Qualification  Trials,  the  Navy 
conducted  integrated  developmental  and  operational  testing 
in  the  air  defense  and  Undersea  Warfare  mission  areas  on 
USS  John  Paul  Jones  (DDG  53),  USS  B enfold  (DDG  65), 
and  USS  Barry  (DDG  52).  Data  from  these  events  will 
supplement  data  collected  during  dedicated  operational  testing 
for  Baseline  9C.  The  Navy  is  scheduled  to  begin  Baseline 

9C  operational  testing  on  USS  John  Paul  Jones  (DDG  53)  in 
FY16. 

•  The  lack  of  an  adequate  modeling  and  simulation  (M&S)  suite 
of  the  Aegis  Combat  System,  as  well  as  the  lack  of  an  Aegis 
equipped  Self-Defense  Test  Ship  (SDTS)  where  the  ship’s  full 
self-defense  kill  chain  can  be  tested,  precludes  assessment  of 
the  Baseline  9  Probability  of  Raid  Annihilation  requirement. 

•  The  Navy  will  not  fully  assess  Aegis  Integrated  Air  and 
Missile  Defense  (IAMD)  until  a  validated  M&S  test  bed 
is  developed  and  validated.  The  test  bed  is  planned  to  be 
available  by  FY20,  but  there  is  no  agreed  upon  strategy  to 
validate  the  model  to  support  assessment  of  the  close-in, 
self-defense  battlespace.  A  limited  IAMD  assessment  will  be 
made  during  Baseline  9C  operational  testing  on  DDGs. 


{ 


•  The  Navy  fielded  the  Navy  Integrated  Fire  Control  -  Counter 
Air  (NIFC-CA)  From-the-Sea  (FTS)  Increment  1  capability 
with  the  deployment  of  the  first  E-2D  and  Baseline  9-equipped 
Carrier  Strike  Group  in  FY15.  NIFC-CA  FTS  Increment  1 
developmental  test  events  in  FY13  and  FY14  demonstrated 

a  basic  capability,  but  its  effectiveness  under  operationally 
realistic  conditions  is  undetermined. 

•  As  discussed  in  the  July  2015  Aegis  Baseline  9A  Early 
Fielding  Report,  DOT&E  is  concerned  with  results  from 
the  cruiser  cybersecurity  evaluation  and  performance  in  the 
Surface  Warfare  mission  area.  Follow-on  cybersecurity  and 
Surface  Warfare  operational  testing  will  be  required. 

System 

•  The  Navy’s  Aegis  Modernization  program  provides  updated 
technology  and  systems  for  existing  Aegis-guided  missile 
cruisers  (CG  47  class)  and  destroyers  (DDG  51  class).  This 
planned,  phased  program  provides  similar  technology  and 
systems  for  new  construction  destroyers. 

•  The  AWS,  carried  on  DDG  5 1  -guided  missile  destroyers 
and  CG  47-guided  missile  cruisers,  integrates  the  following 
components: 

-  AWS  AN/SPY-1  three-dimensional  (range,  altitude,  and 
azimuth)  multi-function  radar 

-  AN/SQQ-89  Undersea  Warfare  suite  that  includes  the 
AN/SQS-53  sonar,  SQR-19  passive  towed  sonar  array 
(DDGs  51  through  78,  CGs  52  through  73),  and  the 
SH-60B  or  MH-60R  helicopter  (DDGs  79  Flight  IIA 
and  newer  have  a  hangar  to  allow  the  ship  to  carry  and 
maintain  its  own  helicopter) 

-  Close-In  Weapon  System 

-  A  5-inch  diameter  gun 
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-  Harpoon  anti-ship  cruise  missiles  (DDGs  51  through  78, 
CGs  52  through  73) 

-  Vertical  Launch  System  that  can  launch  Tomahawk  land 
attack  missiles,  Standard  surface-to-air  missiles,  Evolved 
Seasparrow  Missiles,  and  Vertical  Launch  Anti-Submarine 
Rocket  missiles 

•  The  AWS  on  Baseline  3  USS  Ticonderoga  (CG  47)  class 
cruisers  and  Flight  1  USS  Arleigh  Burke  destroyers  is  being 
upgraded  to  Baseline  9A  and  9C,  respectfully.  Baseline  9  will 
provide  the  following  new  capabilities: 

-  Full  Standard  Missile-6  (SM-6)  integration 

-  LAMD,  to  include  simultaneous  air  defense  and  ballistic 
missile  defense  missions  on  Aegis  destroyers  equipped 
with  the  new  Multi-Mission  Signal  Processor 

-  NIFC-CAFTS  capability 

•  Starting  with  USS  John  Finn  (DDG  113),  the  AWS  on  new 
construction  Aegis-guided  missile  destroyers  is  Baseline  9C. 

Mission 

The  Joint  Force  Commander/Strike  Group  Commander 

employs  AWS-equipped  DDG  51-guided  missile  destroyers  and 

CG-47-guided  missile  cruisers  to  conduct: 


•  Area  and  self-defense  Anti-Air  Warfare  in  defense  of  the 
Strike  Group 

•  Anti-Surface  Warfare  and  Anti-Submarine  Warfare 

•  Strike  Warfare,  when  armed  with  Tomahawk  missiles 

•  Simultaneous  offensive  and  defensive  warfare  operations 

•  Operations  independently  or  in  concert  with  Carrier  or 
Expeditionary  Strike  Groups  and  with  other  joint  or  coalition 
partners 

Major  Contractors 

•  General  Dynamics  Marine  Systems  Bath  Iron  Works  -  Bath, 
Maine 

•  Huntington  Ingalls  Industries  (formerly  Northrop  Grumman 
Shipbuilding)  -  Pascagoula,  Mississippi 

•  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Moorestown,  New  Jersey 


Activity 

•  The  Navy  conducted  Baseline  9A  cruiser  operational  testing 
on  USS  Chancellorsville  in  2QFY15.  One  planned  live  fire 
event  was  deferred  due  to  target  availability,  and  two  of  four 
additional  planned  at-sea  events  were  not  completed  because 
of  test  execution  problems.  These  unexecuted  operational  test 
events  are  currently  scheduled  for  late  1QFY16.  In  FY15, 
the  Navy  also  conducted  a  cybersecurity  assessment  and 
maintenance  demonstration. 

•  In  July  2015  and  November  2015,  DOT&E  submitted 
two  Early  Fielding  Reports  on  Aegis  Baseline  9A  and  9C, 
respectively. 

•  The  Navy  conducted  integrated  developmental  and 
operational  testing  in  the  Undersea  Warfare  mission  area 
on  USS  John  Paul  Jones  and  USS  Benfold  as  part  of  each 
ship’s  Combat  System  Ship  Qualification  Trials  in  1QFY15 
and  2QFY15,  respectively.  Data  from  these  events  will 
supplement  data  collected  during  dedicated  operational  testing 
for  Baseline  9C.  The  Navy  is  scheduled  to  begin  Baseline 

9C  operational  testing  on  USS  John  Paul  Jones  (DDG  53)  in 
FY16. 

•  The  Navy  successfully  conducted  a  live  fire  IAMD  event 
against  threat  representative  cruise  and  ballistic  missile 
surrogates  on  USS  John  Paul  Jones  in  November  2014.  The 
event,  as  conducted,  included  a  less-stressing  scenario  than 
planned  in  the  Aegis  Modernization  Test  and  Evaluation 
Master  Plan,  and  it  resulted  in  only  one,  vice  two,  SM-3 
missiles  being  fired  simultaneously  with  an  SM-2  air  defense 
missile.  This  was  the  only  live  fire  event  available  to  assess 


Baseline  9C’s  ability  to  simultaneously  engage  cruise  missiles 
and  ballistic  missiles. 

•  The  Navy  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  test  plans. 

Assessment 

•  Baseline  9A  and  9C  testing  completed  to  date  was 
not  sufficient  to  support  an  assessment  of  operational 
effectiveness  or  suitability  prior  the  FY15  USS  Normandy 
and  USS  Benfold  deployments.  In  accordance  with  National 
Defense  Authorization  Act  of  2008,  Section  231,  DOT&E 
submitted  Early  Fielding  Reports  for  each  baseline.  Testing 
on  Baseline  9A  and  9C  ships  to  date  suggest  that  area  air 
defense  performance  against  subsonic  and  supersonic 
high-diving  targets  is  consistent  with  historical  performance 
against  comparable  threats;  however,  the  Navy  has  not 

yet  demonstrated  performance  against  more  stressing 
presentations  during  operational  testing.  Operational  testing, 
to  include  more  stressing  presentations,  is  planned  to  continue 
through  FY16. 

•  The  Navy  will  not  fully  assess  Aegis  IAMD  until  an  AWS 
M&S  test  bed  is  developed  and  validated.  The  test  bed  is 
under  development  and  is  planned  to  be  available  by  FY20; 
however,  there  is  no  agreed  upon  strategy  to  validate  the  model 
to  support  assessment  of  the  close-in,  self-defense  battlespace. 
A  limited  Baseline  9C  IAMD  operational  assessment  suggests 
that  DDGs  can  simultaneously  support  limited  air  defense  and 
ballistic  missile  defense  missions,  within  overall  radar  resource 
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constraints.  This  assessment  is  supported  by  a  successful  live 
fire  event,  managed  by  the  Missile  Defense  Agency,  which 
included  simultaneous  live  firing  of  SM-2  and  SM-3  missiles 
against  threat  representative  targets  in  an  1AMD  engagement. 

•  Results  to  date  of  12  live  flight  tests  events  on  Baseline 
9A  and  9C  ships  suggest  that  area  air  defense  performance 
against  single  subsonic  and  supersonic  high-diving  targets  is 
consistent  with  historical  performance  against  comparable 
threats. 

•  Although  not  presented  for  operational  testing,  the  Baseline  9A 
Surface  Warfare  performance,  specifically  counter  high-speed 
surface  threats  in  littoral  waters,  as  demonstrated  during 
developmental  testing,  indicated  no  improvements  over 
previous  Aegis  baseline  operational  test  results.  For  both 
Baseline  9A  and  9C,  these  results  indicate  that  AWS  does  not 
fully  meet  desired  Surface  Warfare  performance  levels. 

•  As  appropriate,  and  until  the  full  capability  may  be 
operationally  tested,  DOT&E  will  provide  periodic  capability 
assessments  to  inform  Navy  and  OSD  leadership,  as  well  as 
Congress,  on  the  progress  of  T&E  of  the  IAMD  mission  area. 

•  Until  an  Aegis-equipped  SDTS  is  available  for  testing,  it  is 
neither  possible  to  characterize  the  self-defense  capabilities  of 
the  Aegis  cruisers  and  destroyers,  nor  possible  to  accredit  an 
M&S  suite  to  determine  if  the  ships  satisfy  their  Probability  of 
Raid  Annihilation  requirements. 

•  The  Navy’s  NIFC-CA  FTS  Increment  I  test  events  conducted 
to  date  are  sufficient  to  demonstrate  basic  capability;  however, 
these  demonstrations  were  not  conducted  under  operationally 
realistic  conditions  or  against  aerial  targets  representative  of 
modern  threats.  Additionally,  the  scenarios  conducted  were 
not  sufficiently  challenging  to  demonstrate  the  NIFC-CA 
FTS  requirements  defined  in  the  Navy’s  September  2012 
NIFC-CA  FTS  Testing  Capability  Definition  Letter.  Further 
testing  is  planned  for  FY16;  these  tests,  too,  will  not  be 
sufficiently  challenging  to  allow  an  operational  effectiveness 
determination. 

•  The  Navy’s  combined  Baseline  9  and  SM-6  FOT&E  test 
events  to  date  have  been  successful  with  no  SM-6  integration 
issues  revealed. 

•  The  Navy’s  Aegis  Baseline  9A  cybersecurity  testing  revealed 
significant  problems,  which  are  classified.  The  nature  of  these 
problems  is  such  that  they  could  pose  significant  risk  to  the 
cybersecurity  for  the  FY15  deployment.  Details  can  be  found 
in  DOT&E ’s  Early  Fielding  Report  dated  July  2015. 

•  Changes  made  to  the  radar  software  presented  unexpected 
issues  during  the  initial  phase  of  the  Aegis  cruiser  at-sea 
operational  test.  The  Navy  is  addressing  these  issues  and 


remaining  cruiser  and  destroyer  operational  testing  will 
provide  opportunities  to  confirm  these  issues  have  been 
mitigated. 

•  During  both  integrated  and  operational  test  events,  instability 
of  the  Aegis  operator  consoles  adversely  affected  the  conduct 
of  test  events.  The  Navy  is  addressing  these  issues  and 
remaining  cruiser  and  destroyer  operational  testing  will 
provide  opportunities  to  confirm  these  issues  have  been 
mitigated. 

•  Aegis  Baseline  9C  has  incorporated  software  changes  to 
address  performance  against  certain  stressing  air  defense  threat 
presentations;  however,  the  effects  of  these  changes  remain 
undemonstrated  by  testing.  Developmental  testing  of  these 
changes  is  planned  for  late  1QFY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  not 
addressed  the  following  previous  recommendations  from 
FY14.  The  Navy  still  needs  to: 

1.  Continue  to  improve  Aegis  ships’  capability  to  counter 
high-speed  surface  threats  in  littoral  waters. 

2.  Synchronize  future  baseline  operational  testing  and 
reporting  with  intended  ship-deployment  schedules  to 
ensure  that  testing  and  reporting  is  completed  prior  to 
deployment. 

3.  Provide  the  necessary  funding  to  support  the  procurement 
of  an  advanced  air  and  missile  defense  radar  and 
Aegis-equipped  SDTS  that  is  needed  to  support  Aegis 
Modernization,  advanced  air  and  missile  defense  radar, 
DDG  51  Flight  III,  and  Evolved  Seasparrow  Missile 
Block  2  operational  testing. 

4.  Characterize  Aegis  Baseline  9A/C  and  NIFC-CA  FTS 
Increment  I  capability  against  operationally  realistic 
anti-ship  cruise  missile  threats  as  soon  as  possible. 

5.  Submit  a  Test  and  Evaluation  Master  Plan  for  DOT&E 
approval  that  describes  and  resources  adequate  operational 
testing  of  future  NIFC-CA  FTS  increments  before  such 
capabilities  are  deployed. 

6.  For  Baseline  9A,  develop  and  deploy  necessary 
cybersecurity  corrective  actions  and  verify  correction  with  a 
follow-on  operational  cybersecurity  test. 

•  FY15  Recommendation. 

1.  The  Navy  needs  to  complete  the  planned  FOT&E  events  as 
detailed  in  the  approved  test  plan  as  soon  as  practical. 
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AGM-88E  Advanced  Anti-Radiation  Guided  Missile 

(AARGM)  Program 


Executive  Summary 

•  The  Advanced  Anti-Radiation  Guided  Missile  (AARGM) 
remains  operationally  suitable,  but  not  operationally  effective 
due  to  multiple  deficiencies  discovered  during  IOT&E  in 
FY11-12. 

•  After  delivery  of  missile  flight  software  version  R2. 1,  Navy 
test  squadrons  VX-3 1  and  VX-9  conducted  integrated  testing 
in  4QF Y 14-1 QF Y 1 5 .  Based  on  deficiencies  discovered 
during  this  first  round  of  integrated  testing  (Phase  1),  testing 
was  halted,  significant  software  updates  were  required,  and 
an  additional  integrated  test  phase  was  introduced  (Phase  la). 
Software  version  R2.2  was  delivered  in  3QFY15  for  Phase  la 
testing,  which  was  conducted  3-4QFY15. 

•  Phase  la  test  deficiencies  required  more  software  changes, 
which  DOT&E,  Commander,  Operational  Test  and  Evaluation 
Force,  and  VX-9  are  currently  analyzing.  If  these  changes 
are  deemed  significant,  or  if  additional  changes  become 
necessary  during  Phase  2  of  integrated  testing,  an  additional 
test  phase  may  be  required  to  produce  the  required  data  to 
assess  test  adequacy,  operational  effectiveness,  and  operational 
suitability. 

•  Several  operational  mission  failures  (OMFs)  occurred  in  Phase 
la.  Based  on  final  scoring,  if  further  OMFs  are  discovered  in 
Phase  2,  reliability  and  performance  data  may  be  insufficient 
to  assess  suitability.  Additional  captive  carry  and  live  fire  tests 
may  be  required  for  adequate  IOT&E. 

•  The  Navy  conducted  two  live  fire  test  events  during  Phase  la, 
both  successfully  engaging  their  targets  and  test  objectives 
were  achieved.  The  first  test  was  against  a  moving  maritime 
target  on  the  Point  Mugu  Sea  Range.  The  second  test  was 
against  a  traditional  air  defense  unit  on  the  China  Lake  range. 

•  There  were  no  dedicated  operational  test  events  scheduled  or 
conducted  during  FY15. 

System 

•  AARGM  supplements  the  AGM-88B/C  High-Speed 
Anti-Radiation  Missile  (HARM)  and  is  specifically  designed 
to  prosecute  targets  that  stop  radiating,  executing  point  to 
point  missions  against  traditional  and  non-traditional  air 
defense  systems.  AARGM  uses  a  new  guidance  section  and  a 
modified  HARM  control  section  and  fins.  The  Navy  intends  to 
employ  AARGM  on  F/A-18C/D/E/F  and  EA-18G  platforms. 

•  AARGM  incorporates  digital  Anti-Radiation  Homing,  a  GPS, 
Millimeter  Wave  guidance,  and  a  Weapon  Impact  Assessment 
transmitter. 


-  Anti-Radiation  Homing  improvements  include  an 
increased  field  of  view  and  increased  detection  range 
compared  to  HARM. 

-  The  GPS  allows  position  accuracy  in  location  and  time. 

-  The  Weapons  Impact  Assessment  capability  allows 
transmission  of  real-time  hit  assessment  via  a  national 
broadcast  data  system. 

-  The  Millimeter  Wave  radar  technology  allows  target 
discrimination  and  guidance  during  the  terminal  flight 
phase. 

•  The  Navy  expects  the  AARGM  Block  1  Upgrade  (a  software 
only  upgrade)  to  deliver  Full  Operational  Capability,  including 
Block  0  capability  improvements  and  software  changes 
to  provide  deferred  capability  requirements  and  address 
deficiencies  identified  during  IOT&E. 

Mission 

Commanders  employ  aircraft  equipped  with  AARGM  to 
conduct  pre-planned,  on-call,  and  time-sensitive  reactive 
anti-radiation  targeting  to  suppress,  degrade,  and  destroy  radio 
frequency-enabled  surface-to-air  missile  defense  systems. 

Major  Contractor 

Orbital/Alliant  Techsystems  -  Northridge,  California 
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Activity 

•  There  were  no  dedicated  operational  test  events  scheduled  or 
conducted  during  FY15. 

•  In  June  2015,  DOT&E  approved  the  AARGM  FOT&E  test 
plan  developed  by  the  Program  Office.  The  test  plan  was 
adequate  to  address  the  testing  of  deferred  capabilities  and 
deficiencies  discovered  during  initial  developmental  test  and 
evaluation  and  IOT&E. 

•  In  FY15,  Phase  1  integrated  testing  continued.  Based  on 
analysis  of  weapon  performance  data,  the  Navy  determined 
that  a  software  update  was  required  and  subsequently  stopped 
the  remaining  captive-carry  and  live  fire  test  events.  During 
this  phase,  VX-3 1  and  VX-9  conducted  four  test  events 
comprised  of  79  captive-carry  test  runs. 

•  In  FY15,  Phase  la  of  integrated  testing  was  conducted 
after  the  software  corrections  of  earlier  deficiencies  were 
completed.  VX-3 1  and  VX-9  conducted  1 1  test  events, 
comprised  of  228  captive-carry  test  runs,  and  2  live  fire  test 
shots  during  this  test  phase.  The  Navy  conducted  the  first 
live  fire  test  against  a  moving  maritime  target  on  the  Point 
Mugu  Sea  Range.  The  second  live  fire  test  was  against  a 
traditional  air  defense  unit  on  the  China  Lake  range. 

•  During  Phase  la,  navigational  errors  were  noted  on  several 
occasions.  The  Navy  believes  they  have  identified  the  cause 
of  the  navigational  errors  and  proposed  software  changes  be 
made  to  fix  these  during  Phase  2  testing. 

•  Phase  2  of  integrated  testing  will  begin  in  the  fall  of  2015. 

Assessment 

•  The  FY15  status  remains  unchanged  from  the  FY14  report. 

•  Based  on  IOT&E  test  data,  AARGM  was  determined  to  be 
operationally  suitable,  but  not  operationally  effective.  The 
details  of  these  deficiencies  are  discussed  in  the  classified 
DOT&E  IOT&E  report  published  in  August  2012. 

•  The  AARGM  program  has  continued  developmental  and 
integrated  testing,  based  on  the  delivery  of  the  R2.2  missile 
flight  software  and  additional  software  modifications  added 


from  Phase  1  testing  deficiencies.  DOT&E  does  not  have 
the  final  test  results  of  Phase  la  testing.  However,  based  on 
interim  Phase  la  test  results,  additional  software  changes  have 
been  made  to  correct  navigational  errors.  If  these  additional 
software  changes  are  considerable,  an  additional  phase  of 
integrated  testing  will  be  required,  including  regression  testing 
of  capabilities  already  tested  in  Phase  1  and  Phase  la. 

•  Several  hardware  and  software  OMFs  have  been  reported 
during  Phase  la  testing.  Depending  upon  final  scoring  of 
these  potential  Phase  la  OMFs  and  any  found  in  Phase  2, 
there  may  not  be  sufficient  flight  hours  remaining  in  the 
test  program  (both  integrated  and  operational  testing)  to 
assess  AARGM  effectiveness  and  suitability.  Additional 
captive-carry  and  live  fire  tests  may  be  required  for  adequate 
IOT&E. 

•  The  overall  test  design  and  identified  resources  should 
provide  a  rigorous  evaluation  of  the  corrections  of 
deficiencies  discovered  in  IOT&E  and  the  deferred  classified 
Key  Performance  Parameter,  which  is  classified.  The  early 
integrated  testing  of  captive-carry  and  live  fire  events  are 
designed  to  provide  insight  and  exposure  to  all  capabilities 
and  conditions.  These  initial  test  events  should  give  an  early 
indication  of  the  performance  of  the  missile  and  stability  of  the 
system. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  addressed  all 
previous  recommendations. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Report  the  results  of  Phase  la  integrated  testing  in  1QFY16. 

2.  Assess  whether  sufficient  data  exist  to  support  independent 
operational  test  requirements  based  on  results  from  Phase  1, 
Phase  la,  and  Phase  2  integrated  testing. 

3.  Coordinate  future  AARGM  operational  test  and  resource 
requirements  with  DOT&E,  as  well  as  ensure  production 
representative  assets  are  used  in  the  integrated  test  phase. 
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AIM-9X  Air-to-Air  Missile  Upgrade 


Executive  Summary 

•  The  Navy  and  Air  Force  originally  began  AIM-9X  Block  II 
IOT&E  (OT-C1)  with  Operational  Flight  Software  (OFS) 

9.311  on  April  27,  2012.  On  July  29,  2013,  the  Program 
Executive  Officer  for  Tactical  Aircraft  Programs  (PEO(T)) 
formally  decertified  AIM-9X  Block  II  due  to  two  major 
deficiencies  discovered  and  documented  during  IOT&E  that 
affected  missile  performance.  These  deficiencies  were  poor 
reliability  of  the  inertial  measurement  unit  and  a  software 
performance  problem.  The  contractor  implemented  an 
improved  production  process  and  updated  the  missile  software 
(OFS  9.313)  to  address  the  two  primary  deficiencies. 

•  The  Services  conducted  IOT&E  of  the  Block  II  missile  with 
OFS  9.313  from  June  2014  through  March  2015.  Testing 
included  19  scored  missile  launches;  captive-carry  testing  to 
examine  acquisition,  tracking,  and  reliability;  and  modeling 
and  simulation. 

•  IOT&E  of  the  AIM-9X  Block  II  hardware  with  OFS  9.313 
demonstrated  that  the  missile  is  effective  against  aircraft 
and  cruise  missile  targets.  Deficiencies  discovered  in  earlier 
OFS  9.311  were  found  to  be  fixed.  Additionally,  testing 
demonstrated  that  the  missile  is  suitable  on  the  F-15  and 
F-16  aircraft  and  not  suitable  on  the  F/A-18  aircraft  due  to  a 
high  number  of  aircraft-related  built-in  test  (BIT)  failures  on 
the  F/A-18. 

•  The  Navy  achieved  Initial  Operational  Capability  of  AIM-9X 
Block  II  on  March  31,  2015,  with  Carrier  Air  Wing  FIVE. 

•  The  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  approved  Full-Rate  Production  (FRP)  via  an 
Acquisition  Decision  Memorandum  dated  August  17,  2015. 

•  The  Air  Force  and  Navy  are  in  the  final  stages  of  test  planning 
to  conduct  AIM-9X  cybersecurity  testing. 

System 

•  AIM-9X  is  the  latest  generation  short-range,  heat-seeking, 
air-to-air  missile.  The  currently  fielded  version  of  the 
Block  1  missile  is  OFS  8.220,  which  includes  limited 
lock-on-after-launch,  full  envelope  off  boresight  capability 
with  a  helmet-mounted  cueing  system,  and  improved  flare 
rejection  performance. 

•  AIM-9X  Block  II  missiles  are  currently  fielded  with 

9.314  software,  which  significantly  builds  from  8.220  software 
with  datalink,  lofted  trajectories,  full  lock-on-after-launch 
capability,  and  improved  high-off  boresight  capability  and 
flare  rejection. 

•  AIM-9X  is  highly  maneuverable,  day/night  capable,  and 
includes  the  warhead,  fuze,  and  rocket  motor  from  the 
previous  AIM-9M  missile. 


•  AIM-9X  added  a  new  imaging  infrared  seeker,  vector 
controlled  thrust,  digital  processor,  and  autopilot.  F-15C/D, 
F-16C/D,  and  F/A-18C/D/E/F  aircraft  are  capable  of 
employing  the  AIM-9X,  with  ongoing  integration  activities 
being  conducted  for  the  F-15E,  F-22,  and  F-35A/B/C. 

•  The  AIM-9X  Block  II  is  the  combination  of  AIM-9X-2 
hardware  and  OFS  9.3 14.  (OFS  9.314  is  the  designation 
following  required  security  requirements  implemented  in  the 
9.313  OFS). 

-  AIM-9X  Block  II  is  the  latest  hardware  version  and 
is  designed  to  prevent  parts  obsolescence  and  provide 
processing  capability  for  the  OFS  9.4  upgrade.  The 
AIM-9X-2  missile  includes  a  new  processor,  a  new  battery, 
an  electronic  ignition  safe/arm  device,  and  the  DSU-41/B 
Active  Optical  Target  Detector  fuze/RF  datalink  assembly. 

-  OFS  9.4  is  a  future  software  upgrade  that  is  intended  to 
add  improved  lock-on-after-launch,  target  re-acquisition, 
improved  fuzing,  and  surface  attack. 

Mission 

Air  combat  units  use  the  AIM-9X  to: 

•  Conduct  short-range  offensive  and  defensive  air-to-air  combat 

•  Engage  multiple  enemy  aircraft  types  with  passive  infrared 
guidance  in  the  missile  seeker 

•  Seek  and  attack  enemy  aircraft  at  large  angles  away  from  the 
heading  of  the  launch  aircraft 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 
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Activity 

•  On  July  29,  2013,  the  AIM-9X  Program  Office  and  Raytheon 
Missile  Systems  implemented  hardware  and  software  solutions 
to  address  the  two  primary  deficiencies  discovered  in  OFS 
9.311  testing  that  led  to  PEO(T)  decertification  of  the  program 
from  testing. 

•  On  June  5,  2014,  the  Navy  completed  an  Operational  Test 
Readiness  Review  and  PEO(T)  re-certified  AIM-9X  Block  II 
with  OFS  9.3 13  for  IOT&E.  DOT&E  approved  a  test  plan 
change  reducing  the  number  of  captive-carry  missions  to 

28  (14  per  Service)  and  removed  one  of  the  17  live  missile 
tests  from  the  originally  approved  IOT&E  plan. 

•  The  Services  conducted  an  IOT&E  of  AIM-9X  Block  II 
with  OFS  9.313  from  June  2014  through  March  2015,  in 
accordance  with  the  DOT&E-approved  test  plan.  Testing 
included  19  scored  flight  tests,  26  captive-carry  acquisition 
and  tracking  missions,  2,402  hours  of  captive  carry  reliability 
data,  and  20,000  modeling  and  simulation  runs. 

•  The  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  approved  FRP  via  an  Acquisition  Decision 
Memorandum  dated  August  17,  2015. 

Assessment 

•  DOT&E  assessed  that  the  AIM-9X  Block  II  missile  with  OFS 
9.313: 

-  Is  effective.  In  flight  testing,  15  of  19  scored  missile 
launches  achieved  a  lethal  intercept.  Captive-carry  testing 
demonstrated  solid  acquisition  and  tracking  performance. 
Demonstrated  missile  reliability  is  on  track  to  meet 
requirements. 


-  Is  suitable  on  the  F-15  and  F-16  aircraft,  but  not  suitable 
on  the  F/A-18  aircraft  due  to  a  high  number  of  BIT  failures 
related  to  F/A-18  software.  In  2,402  hours  of  captive-carry 
reliability  testing,  80  aircraft-related  BIT  failures  occurred, 
all  of  which  were  on  the  F/A-18.  . 

-  Future  F/A-18  software  (H10  planned  for  FY16)  addresses 
the  power  management  errors  behind  the  BIT  failures  that 
led  to  the  not  suitable  assessment  of  Block  II  on  F/A-18. 

•  IOT&E  of  the  Block  II  hardware  with  OFS  9.313 
demonstrated  that  the  two  previous  deficiencies  that  led  to 
decertification  have  been  addressed  successfully. 

•  The  Air  Force  and  Navy  are  in  the  final  stages  of  test  planning 
to  conduct  AIM-9X  cybersecurity  testing. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  in  the 
process  of  completing  the  FY14  recommendation  to  work 
closely  with  DOT&E  and  the  Service  Operational  Test 
Agencies  to  establish  the  plan,  requirements,  and  resources 
for  OFS  9.400  testing,  including  the  associated  Test  and 
Evaluation  Master  Plan  update. 

•  FY15  Recommendations. 

1.  The  Navy  should  verify  that  F/A-18  H10  software  resolves 
the  BIT  problems  that  led  to  a  rating  of  not  suitable. 

2.  The  Services  should  complete  cybersecurity  testing  on  the 
AIM-9X  in  accordance  with  the  August  1,  2014  DOT&E 
policy  memorandum. 
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AN/SQQ-89A(V)15  Integrated  Undersea  Warfare  (USW) 

Combat  System  Suite 


Executive  Summary 

•  Operational  testing  of  the  Advanced  Capability  Build  2011 
(ACB-11)  variant  began  in  FY14  and  is  expected  to  conclude 
in  FY16.  However,  the  Navy  has  not  yet  scheduled  all 
required  IOT&E  events.  The  Navy  completed  limited,  at-sea 
testing  in  FY15  in  conjunction  with  two  fleet-training  events. 

•  In  December  2014,  DOT&E  submitted  a  classified  Early 
Fielding  Report  on  the  ACB-11  variant  of  AN/SQQ-89A(V)15 
Integrated  Undersea  Warfare  Combat  System  Suite.  The 
report  was  submitted  due  to  the  installation  of  the  ACB- 1 1 
variant  on  ships  that  deployed  prior  to  IOT&E.  From  the  data 
collected,  DOT&E  concluded  the  system  demonstrated  some 
capability  to  detect  submarines  and  incoming  U.S.  torpedoes 
in  deep  water.  However,  no  data  were  available  to  assess  its 
capability  in  shallow  water,  an  area  of  significant  interest  due 
to  the  prevalence  of  submarines  operating  in  littoral  regions. 
Also,  no  data  were  available  to  assess  performance  against 
threat  torpedoes. 

System 

•  AN/SQQ-89A(V)15  is  the  primary  Undersea  Warfare  system 
used  aboard  U.S.  Navy  surface  combatants  to  locate  and 
engage  threat  submarines.  AN/SQQ-89A(V)15  is  an  open 
architecture  system  that  includes  biennial  software  upgrades 
(Advanced  Capability  Builds)  and  hardware  upgrades  called 
Technology  Insertions  every  four  years. 

•  AN/SQQ-89A(V)15  uses  active  and  passive  sonar  to  conduct 
Anti-Submarine  Warfare  (ASW)  search.  The  acoustic  energy 
received  is  processed  and  displayed  to  enable  operators  to 
detect,  classify,  localize,  and  track  threat  submarines. 

•  AN/SQQ-89A(V)15  uses  passive  sonar  (including  acoustic 
intercept)  to  provide  early  warning  of  threat  torpedoes. 

•  The  Navy  intends  to  improve  sensor  display  integration  and 
automation,  reduce  false  alerts,  and  improve  onboard  training 
capability  to  better  support  operations  within  littoral  regions 
against  multiple  sub-surface  threats. 

•  The  system  consists  of: 

-  Acoustic  sensors  -  hull-mounted  array,  Multi-Function 
Towed  Array  (MFTA)  TB-37  including  a  towed  acoustic 
intercept  array,  calibrated  reference  hydrophone, 
helicopter,  and/or  ship-deployed  sonobuoys 

-  Functional  segments  used  for  processing  and  displaying 
active,  passive,  and  environmental  data 


-  Interfaces  with  Aegis  Combat  System  for  MK  46  and 
MK  54  torpedo  prosecution  using  surface  vessel  torpedo 
tubes,  Vertical  Launch  Anti-Submarine  Rocket,  or 
SH-60B/MH-60R  helicopters 

•  The  system  is  deployed  on  a  DDG  51  class  destroyer  or  CG  47 
class  cruiser. 

Mission 

•  Maritime  Component  Commanders  employ  surface 
combatants  with  AN/SQQ-89A(V)15  as  escorts  to  high-value 
units  to  protect  against  threat  submarines  during  transit. 

•  Maritime  Component  Commanders  use  surface  combatants 
with  AN/SQQ-89A(V)15  to  conduct  area  clearance  and 
defense,  barrier  operations,  and  ASW  support  during 
amphibious  assault. 

•  Theater  Commanders  use  surface  combatants  with 
AN/SQQ-89A(V)15  to  locate  and  monitor  threat  submarines 
in  theater. 

•  Unit  Commanders  use  AN/SQQ-89A(V)15  to  support  defense 
against  incoming  threat  torpedoes. 

Major  Contractor 

Lockheed  Martin  Mission  Systems  and  Training  -  Manassas, 

Virginia 


Activity 

•  In  January  2013,  DOT&E  sent  a  memorandum  to  the  Acquisition)  outlining  the  need  for  a  threat  torpedo  surrogate 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  to  support  operational  testing  of  the  AN/SQQ-89A(V)15  and 
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requesting  the  Navy’s  plan  to  address  this  need.  In  June  2015, 
DOT&E  sent  a  follow-up  memorandum  that  reiterated  the 
need  for  adequate  torpedo  surrogates  in  operational  testing  and 
identified  that  DOT&E  is  still  waiting  for  the  Navy’s  plan. 

•  In  December  2014,  DOT&E  submitted  a  classified  Early 
Fielding  Report  for  the  ACB-11  variant  of  AN/SQQ-89A(V)15 
Integrated  Undersea  Warfare  Combat  System  Suite.  The 
report  was  submitted  due  to  the  installation  of  the  ACB- 1 1 
variant  on  ships  that  deployed  prior  to  IOT&E. 

•  The  Commander,  Operational  Test  and  Evaluation  Force 
continued  IOT&E  on  the  ACB-11  variant  in  May  2015. 

Testing  was  conducted  in  accordance  with  a  DOT&E -approved 
test  plan  and  included  ASW  transit  search  and  area  search 
operations  using  AN/SQQ-89A(V)  15  onboard  a  DDG  51  class 
destroyer.  Testing  was  conducted  in  conjunction  with  the 
following  two  fleet  events: 

-  Ship  ASW  Readiness  and  Evaluation  Measurement 
180  exercise  in  the  Western  Pacific  that  included  search 
opportunities  against  a  diesel  submarine. 

-  Submarine  Command  Course  44  Anti-Surface  Warfare 
events  at  the  Navy’s  Atlantic  Undersea  Test  and  Evaluation 
Center. 

•  In  September  2015,  the  Navy  completed  a  formal  study 
to  identify  capability  gaps  in  currently  available  torpedo 
surrogates  and  present  an  analysis  of  alternatives  for  specific 
investments  to  improve  threat  emulation  ability. 

•  The  Navy  has  not  yet  scheduled  the  remaining  IOT&E  events. 
Remaining  ACB-11  operational  testing  is  needed  to  understand 
ASW  detection  capability  in  shallow  water  (generally 
defined  as  water  that  is  less  than  100  fathoms  in  depth),  an 
environment  that  was  a  focus  for  system  improvement. 

Assessment 

•  The  final  assessment  of  ACB-11  is  not  complete,  as  testing  is 
expected  to  continue  into  FY16.  DOT&E’s  classified  Early 
Fielding  Report  and  additional  analysis  conducted  in  FY15 
suggest  the  following  regarding  performance: 

-  The  ACB- 1 1  variant  meets  program  performance  metrics 
for  submarine  detection  and  classification  in  deep-water 
environments. 

-  The  ACB- 1 1  variant  demonstrated  some  capability  to 
localize  and  support  prosecution  of  a  threat  submarine  in 
deep  water. 

-  The  ACB- 1 1  variant  does  not  meet  program  performance 
metrics  for  torpedo  detection  as  assessed  against 

U.S.  exercise  torpedoes. 

-  The  ACB-11  variant  is  currently  not  suitable  due  to  low 
operational  availability.  ACB-11  software  reliability 

is  sufficient;  however,  hardware  failures  residted  in 
significant  periods  of  limited  system  capability.  Extensive 


logistic  delays,  particularly  with  the  repair  of  the  MFTA, 
are  the  primary  cause  of  low  operational  availability. 

MFTA  repair,  achieved  by  replacing  the  ship’s  array  with  a 
spare  MFTA  when  the  ship  is  in  port,  was  delayed  by  fleet 
inventory  and  positioning  of  spare  arrays 
-  No  assessment  can  be  made  against  the  smaller  midget  and 
coastal  diesel  submarines  due  to  the  Navy  having  no  test 
surrogates  to  represent  this  prevalent  threat. 

•  The  Navy  study  on  threat  torpedo  surrogates  confirmed 
DOT&E’s  concerns  that  current  torpedo  surrogates  have 
significant  gaps  in  threat  representation  for  operational  testing 
and  the  study  provided  recommendations  for  improving 
current  threat  torpedo  emidation.  However,  the  Navy  has  yet 
to  provide  its  plan  to  provide  realistic  torpedo  surrogates  to 
effectively  characterize  AN/SQQ-89A(V)  15  performance  in 
future  operational  tests.  Improved  torpedo  surrogate  capability 
is  required  to  adequately  evaluate  future  ACB  variants. 

•  Analysis  of  the  few  completed  IOT&E  events  in  shallow  water 
indicates  that  the  ACB-11  variant  has  some  capability  to  detect 
submarines  in  shallow  water.  However,  the  fleet  exercises 
either  did  not  support  the  necessary  ranges  to  assess  detection 
against  system  requirements  or  an  exercise  limitation  excluded 
the  use  of  the  MFTA. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  making 
progress  and  should  continue  to  address  all  previous 
recommendations.  Specific  concerns  include: 

1 .  Develop  and  integrate  high-fidelity  trainers  and  realistic, 
in-water  test  articles  to  improve  training  and  proficiency  of 
operators  in  ASW  search  and  track  of  threat  submarines, 
including  midget  and  coastal  diesel  submarines. 

2.  Pre -position  spare  TB-37  MFTA  and  spare  MFTA  modules 
at  appropriate  forward-operating  ports  to  minimize  logistic 
delays. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Schedule  and  complete  dedicated  IOT&E  to  characterize 
ACB-11  operational  performance  in  shallow  water  and 
assess  cybersecurity  vulnerabilities. 

2.  Revisit  system  requirements  to  ensure  that  funded 
improvement  in  subsequent  ACBs  is  supporting  Navy 
objectives  for  ASW  against  current  and  imminent  threat 
submarines. 

3.  Address  the  four  classified  recommendations  listed  in  the 
December  2014  Early  Fielding  Report. 

4.  Develop  and  execute  a  plan  to  provide  representative 
torpedo  surrogates  before  evaluation  of  the  next  ACB 
development  that  is  focused  on  torpedo  recognition 
capability  (detection  and/or  classification). 
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CH-53K  -  Heavy  Lift  Replacement  Program 


Executive  Summary 

•  In  FY16,  the  CH-53K  program  will  provide  four  Engineering 
Manufacturing  and  Development  (EMD)  aircraft  to  support 
integrated  developmental  and  operational  flight  testing.  A 
Ground  Test  Vehicle  (GTV)  is  being  used  to  qualify  key 
dynamic  components  and  assess  aircraft  stresses,  vibrations, 
and  rotor  performance.  GTV  also  supports  long  term 
verification  and  reliability  testing.  The  program  achieved  first 
flight  of  the  CH-53K  EMD-1  aircraft  on  October  27,  2015. 

•  Gear  box  failures  discovered  in  ground  testing  in  January 
2015,  most  notably  a  quill  shaft  failure  in  the  Main  Gear  Box, 
have  contributed  to  program  delays.  The  quill  shaft  and  the 
Rear  Module  Assembly  of  the  gear  box  have  been  redesigned 
and  testing  has  resumed  on  the  GTV. 

•  Initial  environmental  qualifications  of  the  pilot  armored  seats 
experienced  temperature-induced  cracking  in  the  seat  bucket 
and  wing  armor,  which  has  driven  a  redesign. 

•  The  Navy  reduced  the  cabin  floor  and  sidewall  armor 
requirements  to  allow  for  a  lighter  armor  design.  The  accepted 
design  was  qualified  and  assessed  as  part  of  the  CV-22 
LFT&E  program  in  FY14. 

•  In  FY15,  the  Navy  completed  live  fire  vulnerability  testing  of 
the  fuel  system  and  flight-critical  main  and  tail  rotor  system 
components.  With  one  exception,  preliminary  assessment  of 
the  data  revealed  no  unexpected  vulnerabilities.  The  effect  of 
the  observed  main  and  tail-rotor  combat-induced  damage  on 
aircraft  survivability  will  be  assessed  after  the  cyclic  structural 
testing,  scheduled  for  FY16. 

System 

•  The  CH-53K  is  a  new-build,  fly-by-wire,  dual-piloted, 
three-engine  heavy  lift  helicopter  slated  to  replace  the  aging 
CH-53E.  The  CH-53K  is  designed  to  carry  27,000  pounds 
useful  payload  (three  times  the  CH-53E  payload)  to  a  distance 
of  up  to  110  nautical  miles  and  climbing  from  sea  level  at 
103  degrees  Fahrenheit  to  3,000  feet  above  mean  sea  level  at 
91.5  degrees  Fahrenheit. 

•  The  greater  lift  capability  is  facilitated  by  increased  engine 
power  (7,500  shaft  horsepower  versus  4,380  horsepower 
per  engine  in  the  CH-53E)  and  a  composite  airframe.  This 
composite  airframe  is  lighter  than  the  CH-53E  metal  airframe. 

•  The  CH-53K  design  incorporates  the  following  survivability 
enhancement  features: 


-  Aircraft  Survivability  Equipment  to  include  Large  Aircraft 
Infrared  Countermeasures  with  the  advanced  threat 
warning  sensors  (combines  infrared,  laser,  and  hostile 

fire  functions  into  a  single  system),  AN/APR-39D(V)2 
radar  warning  receiver,  and  AN/ALE-47  countermeasure 
dispensing  system. 

-  Pilot  armored  seats,  cabin  armor  for  the  floor  and 
sidewalls,  fuel  tank  inerting,  self-sealing  fuel  bladders,  and 
30-minute  run  dry  capability  gear  boxes. 

•  The  CH-53K  maintains  a  logistics  shipboard  footprint 

equivalent  to  that  of  the  CH-53E. 

Mission 

•  Commanders  will  employ  the  Marine  Air-Ground  Task  Force 

equipped  with  the  CH-53K  for: 

-  Heavy  lift  missions,  including  assault  transport  of 
weapons,  equipment,  supplies,  and  troops 

-  Supporting  forward  arming  and  refueling  points  and  rapid 
ground  refueling 

-  Assault  support  in  evacuation  and  maritime  special 
operations 

-  Casualty  evacuation 

-  Recovery  of  downed  aircraft,  equipment,  and  personnel 

-  Airborne  control  for  assault  support 

Major  Contractor 

Sikorsky  Aircraft  Corporation  -  Stratford,  Connecticut 


Activity 

•  In  FY16,  the  program  will  provide  four  EMD  aircraft  to 
support  integrated  developmental  and  operational  flight 
testing. 


The  Navy  intends  ongoing  GTV  testing  to  qualify  key 
dynamic  components  and  assess  aircraft  stresses,  vibrations, 
and  rotor  performance.  GTV  also  supports  long  term 
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verification  and  reliability  testing.  The  program  achieved  first 
flight  of  the  CH-53K  EMD-1  aircraft  on  October  27,  2015. 

•  In  previous  testing,  the  Navy  completed  qualification  and  live 
fire  testing  of  the  full-up  and  installed  sponson  fuel  cell  against 
operationally  realistic,  small  arms  threats. 

•  The  pilots’  armored  seats  experienced  thermal  cracking  during 
initial  environmental  qualifications  and  had  to  be  redesigned  in 
FY13.  The  new  design  was  qualified  by  analysis  and  has  been 
part  of  the  qualification  program  to  date.  Final  environmental 
and  live  fire  testing  of  the  redesigned  pilot  seat  armor  against 
the  specification  small  arms  threat  is  scheduled  for  FY16. 

•  In  FY15,  the  Navy  changed  the  requirement  and  design  of 
the  cabin  floor  and  sidewall  armor  to  an  achievable  armor 
performance  given  the  aircraft  weight  constraints.  The  new, 
albeit  reduced,  requirements  allow  for  a  lighter  armor  design 
that  has  already  been  qualified  as  effective  against  specific 
CV-22  ballistic  threats. 

•  In  January  through  August  2015,  the  Navy  completed  live 
fire  testing  of  three  flight-critical  main  and  tail  rotor  system 
components.  Testing  was  conducted  against  a  range  of 
operationally  relevant,  small  arms  threats  and  under  static 
loads  representative  of  flight  conditions.  Sikorsky  will  subject 
the  damaged  components  to  post-ballistic  cyclic  structural 
testing  to  assess  the  residual  flight  capability,  representative 
of  get-home  flight  and  landing  conditions.  This  testing  is 
scheduled  for  FY16. 

•  The  Navy  is  modifying  the  Aircraft  Survivability  Equipment 
to  address  cybersecurity  requirements  (data  at  rest  protection), 
obsolescence  (removable  media  and  computer  processors), 
and  reduce  life  cycle  cost  (elimination  of  components).  The 
Navy  is  upgrading  the  infrared  countermeasure  subsystem  and 
adding  hostile  fire  indication. 

•  The  Program  Office  is  revising  the  Test  and  Evaluation  Master 
Plan  (TEMP)  to  reflect  programmatic  changes  and  updates 

to  the  cybersecurity  test  strategy  for  Milestone  C  to  include  a 
Cooperative  Vulnerability  and  Penetration  Assessment  and  an 
Adversarial  Assessment. 

•  The  Navy  conducted  testing  in  accordance  with  a 
DOT&E-approved  TEMP  and  a  DOT&E-approved  2010 
Alternative  LFT&E  plan. 

Assessment 

•  Component  level  bench  testing  and  GTV  testing  uncovered 
gear  box  failures,  most  notably  a  quill  shaft  failure  in  the  Main 


Gear  Box.  The  quill  shaft  and  the  Rear  Modide  Assembly  of 
the  gear  box  required  redesign  before  testing  could  resume 
on  the  GTV.  New  Rear  Module  Assemblies  were  installed 
on  the  GTV  and  the  EMD  aircraft  and  the  subsequent 
qualification  testing  was  completed  to  enable  first  flight  on 
October  27,  2015. 

•  Preliminary  assessment  of  the  sponson  fuel  cell  qualification 
test  data  indicates  acceptable  performance  against  small  arms 
threats.  Additional  live  fire  ballistic  tests  will  be  performed  on 
the  GTV  in  FY19. 

•  During  initial  environmental  qualification  testing,  the  armored 
pilot  seat  did  not  fully  meet  environmental  specifications, 
experiencing  some  thermal  cracking.  The  Program  Office 
initiated  redesign  of  the  seats,  and  final  environmental  testing, 
as  well  as  live  fire  testing  of  the  redesigned  pilot  seat  armor 
against  the  specification  small  arms  threat,  is  scheduled  for 
FY16. 

•  Three  of  the  four  flight-critical  main  and  tail  rotor  system 
components  tested  to  date  demonstrated  the  required  ballistic 
damage  tolerance  to  the  specified  projectile.  The  Navy  will 
assess  the  consequent  effect  of  the  observed  damage  on  aircraft 
survivability,  in  operationally  representative  conditions,  after 
the  completion  of  the  respective  structural  cyclic  endurance 
tests. 

•  Modification  of  the  Aircraft  Survivability  Equipment 
accelerates  inclusion  of  additional  capabilities  while  reducing 
life  cycle  costs. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY 15  Recommendations.  The  program  should: 

1.  Update  the  TEMP  in  FY16. 

2.  Review  data  resulting  from  a  DOT&E  funded  joint  live  fire 
program  to  assess  CV-22  armor  performance  against  threats 
that  the  Navy  did  not  address  in  the  CV-22  Advanced 
Ballistic  Stopping  System  LFT&E  program.  This  will 
enable  the  Navy  to  better  understand  the  effectiveness  of  the 
armor  against  additional,  operationally  realistic  threats  and 
adjust  the  tactics,  techniques,  and  procedures,  as  needed. 
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Common  Aviation  Command  and  Control  System 

(CAC2S) 


Executive  Summary 

•  In  October  2014,  the  Marine  Corps  Operational  Test  and 
Evaluation  Activity  conducted  an  operational  assessment  (OA) 
for  the  Common  Aviation  Command  and  Control  System 
(CAC2S)  Increment  I  Phase  2  during  the  Weapons  and  Tactics 
Instructors’  (WTI)  exercise  at  Marine  Corps  Air  Station 
Yuma,  Arizona.  The  OA  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan. 

•  During  the  OA,  CAC2S  Increment  I  Phase  2  demonstrated 
the  ability  to  support  mission  accomplishment  of  the  three 
Marine  Corps  aviation  command  and  control  agencies. 
Additionally,  CAC2S  demonstrated  the  ability  to  provide 
data  fusion  of  real-time,  near  real-time,  and  non  real-time 
information  onto  a  single  tactical  display. 

•  In  2QFY15,  the  Assistant  Secretary  of  the  Navy,  Research, 
Development,  and  Acquisition,  as  the  Milestone  Decision 
Authority,  conducted  a  Milestone  C  review  for  CAC2S, 
which  resulted  in  an  approval  to  enter  the  Production  and 
Deployment  Phase  of  its  lifecycle  and  to  procure  low-rate 
initial  production  items  to  support  IOT&E. 

•  During  3QFY15  and  4QFY15,  the  Marine  Corps  conducted 
additional  data  fusion  testing  using  updated  operational 
scenarios,  and  integrated/interoperability  testing  with  the 
Composite  Tracking  Network  (CTN).  At  the  end  of  FY15,  the 
Marine  Corps  continued  risk  reduction  efforts  by  conducting 

a  full  Tactical  Air  Command  Center  (TACC)  functionality 
demonstration  during  a  WTI  exercise  at  Marine  Corps  Station 
Yuma,  Arizona,  as  well  as  datalink  testing  and  an  integration 
demonstration  with  the  Ground/Air  Task  Oriented  Radar 
(G/ATOR). 

•  IOT&E  for  CAC2  Increment  I  Phase  2  is  scheduled  for 
3QFY16. 

System 

•  CAC2S  consists  of  tactical  shelters,  software,  and  common 
hardware.  The  hardware  components  are  expeditionary, 
common,  modular,  and  scalable.  Components  may  be 
freestanding,  mounted  in  transit  cases,  or  rack-mounted  in 
shelters  and/  or  general-purpose  tents  that  are  transported  by 
organic  tactical  mobility  assets. 

•  CAC2S  Increment  I  is  being  delivered  in  two  phases.  Phase  I 
previously  delivered  hardware  and  software  to  fully  support 
the  Direct  Air  Support  Center  (DASC)  mission  requirements 
and  partially  support  Tactical  Air  Operations  Center  (TAOC) 
mission  requirements.  Phase  2  combines  the  three  legacy 
Phase  1  systems  into  two  functional  subsystems  and  fully 
supports  the  requirements  of  the  DASC,  TACC,  and  TAOC. 


HF  -  High  Frequency 

JREAP  -  Joint  Range  Extension  Application  Protocol 

MIDS  -  Multi-Functional  Information  Distribution  System 

OPFAC  -  Operations  Facility 

SATCOM  -  Satellite  Communications 

TDL  -  Tactical  Data  Link 

UHF  -  Ultra  High  Frequency 


-  The  Communication  Subsystem  provides  the  capability  to 
interface  with  internal  and  external  communication  assets 
and  the  means  to  control  their  operation. 

-  The  Aviation  Command  and  Control  System  provides: 

■  The  operational  command  post  and  functionality 
to  support  mission  planning,  decision  making,  and 
execution  tools  to  support  all  functions  of  Marine 
Aviation 

■  An  open  architecture  interface  capable  of  integrating 
emerging  active  and  passive  sensor  technology  for 
organic  and  non-organic  sensors  to  the  Marine  Air 
Command  and  Control  System 

■  The  capability  to  display  real-time,  near  real-time, 
and  non  real-time  sensor  data  to  support  C2  of 
Marine  Air-Ground  Task  Force  (MAGTF)  aviation  assets 


Mission 

•  The  MAGTF  Commander  will  employ  Marine  Corps  aviation 
C2  assets,  including  the  DASC,  the  TAOC,  and  the  TACC 
equipped  with  CAC2S,  to  integrate  Marine  Corps  aviation 
into  joint  and  combined  air/ground  operations  in  support  of 
Operational  Maneuver  from  the  Sea,  Sustained  Operations 
Ashore,  and  other  expeditionary  operations. 

•  The  MAGTF  Commander  will  execute  C2  of  assigned  assets 
afloat  and  ashore  in  a  joint,  allied,  or  coalition  operational 
environment  by  using  CAC2S  capabilities  to: 
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Share  mission-critical  voice,  video,  sensor,  and  C2  data 
and  information  to  integrate  aviation  and  ground  combat 
planning  and  operations 

Display  a  common,  real,  and  near  real-time  integrated 
tactical  picture  with  the  timeliness  and  accuracy  necessary 
to  facilitate  the  control  of  friendly  assets  and  the 
engagement  of  threat  aircraft  and  missiles 
Provide  fusion  of  real-time,  near  real-time,  and  non 
real-time  information  to  support  the  MAGTF 
Access  theater  and  national  intelligence  sources  from  a 
multi-function  C2  node 

Standardize  Air  Tasking  Order  and  Airspace  Control 
Order  generation,  parsing,  interchange,  and  dissemination 


throughout  the  MAGTF  and  theater  forces  by  using  the 
joint  standard  for  Air  Tasking  Order  interoperability 

Major  Contractors 

•  Phase  1 

-  Government  Integrator:  Naval  Surface  Warfare 
Center  -  Crane,  Indiana 

-  Component  Contractor:  Raytheon-Solipsys  -  Fulton, 
Maryland 

•  Phase  2 

-  Prime  Contractor  (no  Government  Integrator):  General 
Dynamics  -  Scottsdale,  Arizona 


Activity 

•  In  October  2014,  the  Marine  Corps  conducted  an  OA  of 
the  CAC2S  Increment  1  Phase  2  in  accordance  with  a 
DOT&E-approved  OA  plan. 

•  In  2QFY15,  the  Assistant  Secretary  of  the  Navy,  Research, 
Development,  and  Acquisition,  as  the  Milestone  Decision 
Authority,  conducted  a  Milestone  C  review  for  CAC2S,  which 
resulted  in  an  approval  to  procure  low-rate  initial  production 
items  to  support  IOT&E. 

•  During  May  2015,  the  Program  Office  conducted 
interoperability/integration  testing  with  the  CTN. 

•  In  June  2015,  and  again  in  August  2015,  the  Marine 
Corps  conducted  data  fusion  testing  using  an  updated  and 
operationally  realistic  scenario  that  more  adequately  stressed 
the  system. 

•  The  Marine  Corps  conducted  CTN  connectivity  testing  during 
3QFY15  and  continued  CTN  evaluation  during  the  fall  2015 
WTI  course. 

•  The  Marine  Corps  continued  risk  reduction  efforts  with  a 
datalink  connectivity  functionality  demonstration  during 
4QFY15. 

•  During  the  fall  2015  WTI  course,  the  Program  Office 
conducted  integrated  testing  of  CAC2S  for  all  operations  cells 
within  the  TACC.  In  addition,  operational  endurance  testing 
was  conducted  over  the  same  period  as  risk  reduction  for  the 
upcoming  IOT&E.  An  integration  demonstration  of  CAC2S 
with  G/ATOR  was  also  conducted  during  the  WTI  primarily  as 
a  risk  reduction  effort  since  the  G/ATOR  system  is  still  under 
development. 

•  IOT&E  for  CAC2S  Increment  I  Phase  2  is  scheduled  for 
3QFY16. 

Assessment 

•  Based  on  qualitative  evaluation  during  the  1QFY14  OA: 

-  CAC2S  successfully  demonstrated  the  ability  to  support 
the  primary  mission  areas  for  all  three  agencies:  direct 
air  support  for  the  DASC,  control  aircraft  and  missiles  for 
the  TAOC,  and  C2  aviation  and  planning  support  for  the 
MAGTF  commander  in  the  TACC. 


-  CAC2S  demonstrated  an  ability  to  fuse  real-time,  near 
real-time,  and  non  real-time  data  onto  a  single  tactical 
display,  at  medium  operational  tempo  densities  of  aircraft 
and  targets  against  older/current  generation  threats. 

-  DOT&E  did  not  assess  interoperability/integration  of 
CAC2S  with  G/ATOR  as  that  system  is  still  undergoing 
development.  However,  testing  did  demonstrate  the  ability 
to  connect  the  AN/TPS-59  radar  sensor  directly  to  CAC2S 
displaying  both  radar  plot  and  track  data. 

-  Reliability,  availability  and  maintainability  data,  collected 
during  testing  and  throughout  the  remainder  of  FY15, 
indicate  that  CAC2S  continues  to  make  progress  toward 
meeting  its  reliability  objectives. 

•  During  3QFY15  and  4QFY15,  the  Marine  Corps  continued 
testing  the  CAC2S  data  fusion  capability,  successfully 
demonstrating  the  ability  to  fuse  real-time,  near  real-time, 
and  non  real-time  data  against  an  updated  operational  threat 
scenario  for  the  test  venue. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Marine  Corps 
addressed  some  previous  problems  and  is  in  the  process  of 
addressing  the  remaining  recommendations: 

1.  Utilize  a  balanced  use  of  air  and  ground  combat  forces 
during  future  test  venues  to  provide  a  better  assessment  of 
CAC2S  support  to  the  MAGTF. 

2.  Conduct  24-hour  operations  to  ensure  adequate  hours  for 
assessment  of  system  reliability. 

3.  Conduct  interoperability  and  integration  testing  with  CTN 
and  G/ATOR  in  an  operationally  realistic  environment 
prior  to  IOT&E  in  order  to  reduce  risk  if  those  systems  are 
sufficiently  mature. 

4.  Conduct  a  Field  User  Evaluation  prior  to  IOT&E  that 
exercises  all  di visions/ sections  within  the  TACC. 

•  FY 15  Recommendations.  The  Marine  Corps  should: 

1.  Continue  data  fusion  testing  in  support  of  the  CAC2S 
IOT&E  in  FY16.  Data  fusion  testing  must  be  conducted 
using  operationally  realistic  scenarios  with  the  most 
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likely  air  threat  available  in  a  stressing  operational  tempo 
environment. 

2.  Conduct  integration/interoperability  testing  of  CAC2S 
with  G/ATOR  in  order  to  assess  system  characteristics  and 
support  integration  of  G/ATOR  when  that  system  achieves 
its  Initial  Operational  Capability. 

3.  Complete  a  user  evaluation  of  all  system  functionality 
during  a  TACC  support  demonstration.  Ensure  that 
evaluations  include  those  for  the  Future  Plans,  Future 


Operations,  and  Air  Combat  Intelligence  cells  in  addition  to 
the  Current  Operations  cell  in  order  to  reduce  risk  prior  to 
IOT&E. 

4.  Conduct  datalink  demonstration  testing,  and  where  feasible, 
use  data  in  support  of  CAC2S  IOT&E  scheduled  for 
3QFY16.  Ensure  that  functionality  and  system  usability  are 
assessed  as  part  of  the  pre-operational  test  risk  reduction 
effort. 
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Consolidated  Afloat  Networks  and  Enterprise  Services 

(CANES) 


Executive  Summary 

•  The  Commander,  Operational  Test  and  Evaluation  Force 
(COTF)  completed  the  Consolidated  Afloat  Networks  and 
Enterprise  Services  (CANES)  IOT&E  for  the  unit-level 
variant  on  USS  Higgins  (DDG  76)  from  August  2014  through 
March  2015.  COTF  suspended  the  test  at  the  request  of 

the  program  manager  to  allow  correction  of  cybersecurity 
vulnerabilities. 

•  DOT&E  assessed  the  unit-level  variant  as  operationally 
effective,  suitable,  and  survivable.  CANES  provides 
computing  resources;  throughput  and  transfer  speed;  and 
reliability,  availability,  and  maintainability  necessary  for 
the  ship’s  missions;  however,  the  Navy  should  address 
deficiencies  related  to  training,  network  mapping,  and 
emissions  control  conditions.  The  USD(AT&L)  approved  the 
Full  Deployment  Decision  (FDD)  on  October  13,  2015. 

•  COTF  began  the  FOT&E  of  the  force-level  CANES  variant 
on  the  USS  John  C.  Stennis  (CVN  74)  in  July  2015.  The  test 
will  conclude  in  2016  with  cybersecurity  testing.  DOT&E’s 
initial  observations  are  that  the  force-level  variant  will  provide 
the  network  and  enterprise  capability  to  enable  the  crew  to 
complete  assigned  missions  and  tasks. 

•  The  Navy  plans  to  conduct  an  FOT&E  for  the  submarine 
variant  in  FY18. 

System 

•  CANES  is  an  enterprise  information  system  consisting 
of  computing  hardware,  software,  and  network  services 
(e.g.,  phone,  email,  chat,  video  teleconferencing,  web  hosting, 
file  transfer,  computational  resources,  storage,  and  network 
configuration  and  monitoring).  CANES  will  replace  legacy 
networks  on  ships,  submarines,  and  shore  sites. 

•  The  CANES  program  will  mitigate  hardware  and  software 
obsolescence  on  naval  vessels  through  the  increased  use  of 
standard  components  and  regularly  scheduled  hardware  and 
software  updates. 


•  The  CANES  network  provides  a  single,  consolidated  physical 
network  with  logical  sub-networks  for  Unclassified,  Secret, 
Secret  Releasable,  and  Top  Secret  security  domains.  It 
includes  a  cross-domain  solution  for  information  transfers 
across  these  security  boundaries.  This  consolidation  is 
expected  to  reduce  the  network  infrastructure  footprint  on 
naval  platforms  and  the  associated  logistics,  sustainment,  and 
training  costs. 

Mission 

Shipboard  users  will  use  the  CANES  network  to: 

•  Flost  their  applications  in  support  of  naval  and  joint  operations 
with  computing  resources  and  networks  services 

•  Support  weapon  systems,  command  and  control,  intelligence, 
and  business  information  applications 

Major  Contractors 

•  Northrop  Grumman  -  Huntsville,  Alabama 

•  BAE  Systems  -  Rockville,  Maryland 

•  Serco  -  Reston,  Virginia 


Activity 

•  COTF  completed  IOT&E  for  the  CANES  unit-level  variant 
onboard  USS  Higgins  from  August  2014  through  March  2015. 
COTF  suspended  testing  at  the  program  manager’s  request  to 
allow  for  correction  of  cybersecurity  vulnerabilities. 

•  In  July  2015,  DOT&E  submitted  an  IOT&E  report  of  the 
unit-level  variant  ships,  detailing  the  results  of  testing 

to  inform  the  FDD.  USD(AT&L)  approved  the  FDD  on 
October  13,  2015. 


•  The  CANES  Program  Office  conducted  an  integrated  test  of 
the  CANES  force-level  variant  aboard  USS  John  C.  Stennis 
in  June  2015.  COTF  conducted  a  maintenance  demonstration 
concurrently. 

•  COTF  began  FOT&E  on  the  force-level  variant  in  July  2015 
aboard  USS  John  C.  Stennis.  The  test  period  will  end  after 
cybersecurity  testing  in  2016. 
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Assessment 

•  DOT&E  assessed  the  CANES  unit-level  variant  as 
operationally  effective,  suitable,  and  survivable.  Network 
performance  among  all  security  enclaves  enabled  the  crew 
to  complete  assigned  tasks  and  missions  while  pier-side  and 
underway.  Hardware  and  software  met  reliability,  availability, 
and  maintainability  requirements. 

•  The  program  manager  successfully  mitigated  cybersecurity 
vulnerabilities,  including  working  with  the  fleet  personnel  to 
implement  recommended  procedures  and  security  updates. 

•  Integrated  testing  of  the  CANES  force-level  variant  in 
June  2015  demonstrated  the  system’s  ability  to  meet  basic 
functional  and  performance  requirements. 


•  DOT&E’s  initial  observations  of  the  FOT&E  for  the 
force-level  variant  are  that  the  variant  provided  the  required 
communications  and  information  technology  support  that 
enabled  the  crew  to  complete  assigned  tasks  and  missions. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  addressed  all 
previous  recommendations. 

•  FY15  Recommendation. 

1.  The  Navy  should  begin  planning  the  FOT&E  for  the 
submarine  variant  scheduled  for  FY18. 
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Cooperative  Engagement  Capability  (CEC) 


Executive  Summary 

•  In  July  2015,  DOT&E  submitted  a  classified  Early  Fielding 
Report  on  the  Aegis  Baseline  9A  Combat  System  with  the 
Cooperative  Engagement  Capability  (CEC)  USG-2B.  In  the 
report,  DOT&E  stated  that  test  results  to  date  showed  the 
CEC  USG-2B,  as  integrated  in  the  Aegis  Baseline  9A  Combat 
System,  is  likely  to  perform  comparably  to  previous  CEC 
USG-2  and  USG-2A  variants. 

•  DOT&E  will  conduct  a  full  assessment  of  the  CEC  USG-2B’s 
operational  effectiveness  and  suitability  on  Aegis  platforms 
upon  completion  of  the  CEC  USG-2B  FOT&E. 

System 

•  CEC  is  a  real-time,  sensor-netting  system  that  enables  high 
quality  situational  awareness  and  integrated  fire  control 
capability. 

•  There  are  four  major  U.S.  Navy  variants  of  CEC: 

-  The  USG-2/2A  is  used  in  selected  Aegis  cruisers  and 
destroyers,  LPD  17/LHD  amphibious  ships,  and  CVN  68 
class  aircraft  carriers. 

-  The  USG-2B,  an  improved  version  of  the  USG-2/2A, 
is  used  in  selected  Aegis  cruisers/destroyers  as  well  as 
selected  amphibious  assault  ships.  The  USG-2B  is  planned 
for  use  in  the  CVN  78  and  DDG  1000  ship  classes. 

-  The  USG-3  is  used  in  the  E-2C  Hawkeye  2000  aircraft. 

-  The  USG-3B  is  used  in  the  E-2D  Advanced  Hawkeye 
aircraft. 

•  The  two  major  hardware  pieces  are  the  Cooperative 
Engagement  Processor,  which  collects  and  fuses  sensor  data, 
and  the  Data  Distribution  System,  which  exchanges  data 
between  participating  CEC  units. 

•  The  CEC  increases  Naval  Air  Defense  capabilities  by 
integrating  sensors  and  weapon  assets  into  a  single,  integrated, 
real-time  network  that: 

-  Expands  the  battlespace 

-  Enhances  situational  awareness 


-  Increases  depth-of-fire 

-  Enables  longer  intercept  ranges 

-  Improves  decision  and  reaction  times 

Mission 

•  Naval  forces  use  CEC  to  improve  battle  force  air  and  missile 
defense  capabilities  by  combining  data  from  multiple  battle 
force  air  search  sensors  on  CEC-equipped  units  into  a  single, 
real-time,  composite  track  picture. 

•  Naval  surface  forces  also  use  CEC  to  provide  accurate  air  and 
surface  threat  tracking  data  to  ships  equipped  with  the  Ship 
Self-Defense  System. 

Major  Contractor 

Raytheon  Systems  Co.,  Command,  Control  and  Communications, 
Data  Systems  -  St.  Petersburg,  Florida 


Activity 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  conducted  FOT&E  of  the  CEC  USG-2B  with  the 
Aegis  Baseline  9 A  Combat  System  in  February  2015  in 
accordance  with  a  DOT&E-approved  test  plan.  Problems  with 
test  range  and  aerial  target  availability  delayed  completing  the 
FOT&E  until  January  2016. 

•  DOT&E  submitted  a  classified  Early  Fielding  Report  to 
Congress  on  the  Aegis  Baseline  9A  Combat  System  with  the 
USG-2B  Cooperative  Engagement  Capability  in  July  2015. 


Assessment 

•  Test  results  to  date  indicate  that  a  number  of  effectiveness 
measures  are  below  established  performance  goals.  These 
measures  included  anti-jamming  resistance,  two  data 
distribution  measures,  track  continuity,  identification  accuracy, 
and  interoperability. 

•  In  the  classified  July  2015  Early  Fielding  Report  on  the 
Aegis  Baseline  9A  Combat  System,  DOT&E  stated  that  test 
results  to  date  showed  that  the  CEC  USG-2B,  as  integrated  in 
the  Aegis  Baseline  9A  Combat  System,  is  likely  to  perform 
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comparably  to  previous  CEC  USG-2  variants.  DOT&E  will 
conduct  a  full  assessment  of  the  CEC  USG-2B’s  operational 
effectiveness  and  suitability  upon  completion  of  the  CEC 
USG-2B  FOT&E. 

•  The  CEC  USG-2B  cybersecurity  testing  conducted  to  date  has 
not  revealed  any  major  deficiencies. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  not 
satisfied  the  following  previous  recommendations  to: 

1 .  Demonstrate  corrections  to  the  problem  that  degrades 
the  USG-3B  CEC’s  Track  File  Concurrence  in  a  phase  of 
FOT&E. 

2.  Implement  changes  to  the  USG-3B  CEC  interface  with 
the  E-2D  mission  computer  that  would  allow  data  from 
the  E-2D’s  APY-9  radar  to  be  used  by  the  USG-3B  CEC 
without  first  requiring  the  creation  of  an  E-2D  Mission 
Computer  track. 

3.  Reassess  the  USG-3B  CEC  reliability  requirement  and 
whether  the  logistic  supply  system  can  support  the 
demonstrated  USG-3B  CEC  reliability. 

4.  Correct  the  cause  of  the  electromagnetic  interference 
between  the  USG-3B  CEC  and  the  E-2D  radar  altimeter  and 
demonstrate  the  corrections  in  a  phase  of  FOT&E. 


5.  Take  action  on  the  recommendations  contained  in 

DOT&E ’s  classified  report  to  Congress  on  the  CEC 

USG-3B  FOT&E. 

6.  Update  the  CEC  Test  and  Evaluation  Master  Plan  to  include 

details  of: 

■  The  second  phase  of  the  USG-3B  FOT&E  with  the 
supersonic  sea-skimming  target  scenario 

■  FOT&E  of  corrections  made  to  the  CEC  USG-3B 

■  FOT&E  of  the  CEC  USG-2B  with  the  Aegis  Baseline  9 
Combat  System 

■  FOT&E  of  the  CEC  USG-2B  with  the 
DDG  1000  Zumwalt  Combat  System 

■  FOT&E  of  the  CEC  USG-2B  with  the  CVN  78  Combat 
System 

■  FOT&E  of  USG-3B  CEC  to  demonstrate  the  system’s 
ability  to  support  the  E-2D’s  Theater  Air  and  Missile 
Defense  and  Battle  Force  Command  and  Control 
missions 

■  The  test  program  supporting  the  Acceleration  of 
Mid-term  Interoperability  Improvements  Project 

FY15  Recommendation. 

1.  The  Navy  should  complete  the  FOT&E  of  the  CEC 

USG-2B  with  the  Aegis  Baseline  9  Combat  System. 
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CVN  78  Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier 


Executive  Summary 

•  On  February  2,  2015,  DOT&E  disapproved  Test  and 
Evaluation  Master  Plan  (TEMP)  1610,  Revision  C  because 
the  CVN  78  Class  Full  Ship  Shock  Trial  (FSST)  had  been 
changed  in  the  approved  2007  TEMP  1610,  Revision  B  from 
CVN  78  to  CVN  79.  The  Revision  C  TEMP  does  provide 
improved  integrated  platform-level  developmental  testing, 
reducing  the  likelihood  that  platform-level  problems  will  be 
discovered  during  IOT&E.  In  addition,  the  Program  Office  is 
in  the  process  of  refining  the  post-delivery  schedule  to  further 
integrate  testing. 

•  On  August  7,  2015,  the  Deputy  Secretary  of  Defense  directed 
the  Navy  to  complete  the  FSST  before  CVN  78’s  first 
operational  deployment.  The  Navy  is  updating  the  TEMP  to 
reflect  the  Deputy  Secretary  of  Defense’s  decision. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  began  a  new  DOT&E-approved  operational 
assessment  in  September  2015,  which  is  planned  to  end  in 
mid-2016  after  CVN  78  completes  Builder’s  Sea  Trials  and 
Acceptance  Trials. 

•  DOT&E ’s  assessment  of  CVN  78  remains  consistent  with 
the  DOT&E  Operational  Assessment  report  submitted  in 
December  2013.  Poor  or  unknown  reliability  of  newly 
designed  catapults,  arresting  gear,  weapons  elevators,  and 
radar,  which  are  all  critical  for  flight  operations,  could  affect 
CVN  78’s  ability  to  generate  sorties,  make  the  ship  more 
vulnerable  to  attack,  or  create  limitations  during  routine 
operations.  The  poor  or  unknown  reliability  of  these  critical 
subsystems  is  the  most  significant  risk  to  CVN  78. 

-  Reliability  for  the  catapults  was  last  reported  in 
December  2014.  While  catapult  reliability  is  above  the 
re-baselined  reliability  growth  curve,  the  re-baselined 
curve  is  well  below  the  reliability  requirement  and  the 
catapults  are  unlikely  to  achieve  required  reliability. 

-  Reliability  for  the  arresting  gear  has  not  been  reported  in 
almost  two  years.  The  last  reported  reliability  estimates 
for  the  arresting  gear  were  well  below  the  re-baselined 
reliability  growth  curve,  and  indicated  that  the  system  was 
unlikely  to  achieve  required  reliability.  The  Navy  began 
measuring  reliability  again  in  4QFY15,  but  does  not  expect 
to  have  new  reliability  estimates  until  the  end  of  2015. 
Additionally,  reliability  test  data  are  not  available  for  the 
radar  and  the  weapons  elevators. 

-  Absent  a  major  redesign,  the  catapults  and  arresting  gear 
are  not  likely  to  meet  reliability  requirements. 

•  In  FY14,  testing  at  the  Electromagnetic  Aircraft  Launching 
System  (EMALS)  functional  demonstration  test  site  at  Joint 
Base  McGuire-  Dix-  Lakehurst,  New  Jersey,  discovered 
excessive  airframe  stress  during  launches  of  F/A-18E/F  and 
EA-18G  with  wing-mounted  480-gallon  external  fuel  tanks 
(EFTs).  This  discovery,  until  corrected,  will  preclude  the 


Navy  from  conducting  normal  operations  of  the  F/A-18E/F 
and  EA-18G  from  CVN  78. 

•  In  FY15,  the  Navy  identified  an  inability  to  readily 
electrically  isolate  EMALS  components  to  perform  concurrent 
maintenance.  This  inability  to  readily  electrically  isolate 
EMALS  components  could  preclude  some  types  of  EMALS 
maintenance  during  flight  operations,  decreasing  EMALS 
operational  availability. 

•  In  October  2015,  the  Navy  discovered  that  one  of  the 

three  Prime  Power  Interface  Subsystems  (PPIS)  Transformer 
Rectifiers  (TRs)  had  been  damaged  during  shipboard 
certification  testing.  Two  of  the  three  TRs  are  required 
for  normal  catapult  operations.  The  TRs  were  designed  to 
last  the  life  of  the  ship.  Earlier  faults  discovered  during 
developmental  testing  resulted  in  stepwise  improvements  to 
the  PPIS  TR  design  and  construction.  This  failed  TR  had  one 
of  the  four  improvements. 

•  In  FY15,  the  Navy  began  performance  testing  of  the 
Advanced  Arresting  Gear  (AAG)  at  a  jet  car  track  site  at  Joint 
Base  McGuire-Dix-Lakehurst,  New  Jersey.  This  testing  is 
examining  the  performance  of  the  redesigned  arresting  gear  to 
meet  the  system  specification  with  improve  reliability. 

•  The  CVN  78  design  is  intended  to  reduce  manning.  As 
manning  requirements  have  been  further  developed,  analysis 
indicates  the  ship  is  sensitive  to  manpower  fluctuations;  and 
workload  estimates  for  the  many  new  technologies  such  as 
catapults,  arresting  gear,  radar,  and  weapons  and  aircraft 
elevators  are  not  well-understood.  Some  of  these  concerns 
have  already  required  re-designation  of  some  berthing  areas 
and  may  require  altering  standard  manpower  strategies  to 
ensure  mission  accomplishment. 

•  The  CVN  78  combat  system  for  self-defense  is  derived  from 
the  combat  system  on  current  carriers  and  is  expected  to  have 
similar  capabilities  and  limitations.  The  ship’s  Dual  Band 
Radar  (DBR)  is  being  integrated  with  the  combat  system 
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and  is  undergoing  developmental  testing  at  Wallops  Island, 
Virginia.  That  testing  has  uncovered  significant  problems, 
typical  of  those  seen  in  early  developmental  testing,  affecting 
air  traffic  control  and  self-defense  operations.  The  Navy  is 
investigating  solutions  to  these  problems. 

•  It  is  unlikely  that  CVN  78  will  achieve  its  Sortie  Generation 
Rate  (SGR)  (number  of  aircraft  sorties  per  day)  requirement. 
The  threshold  requirement  is  based  on  unrealistic  assumptions 
including  fair  weather  and  unlimited  visibility,  and  that  aircraft 
emergencies,  failures  of  shipboard  equipment,  ship  maneuvers, 
and  manning  shortfalls  will  not  affect  flight  operations. 
DOT&E  plans  to  assess  CVN  78  performance  during  IOT&E 
by  comparing  it  to  the  demonstrated  performance  of  the  Nimitz 
class  carriers  as  well  as  to  the  SGR  requirement. 

•  CVN  78  will  include  a  new  Heavy  underway  replenishment 
(UNREP)  system  that  will  transfer  cargo  loads  of  up  to 
12,000  pounds.  Currently,  only  one  resupply  ship  has  Heavy 
UNREP  on  one  station.  The  Navy  plans  to  install  a  single 
Heavy  UNREP  station  on  each  additional  resupply  ship 
beginning  in  FY21  with  T-AO(X). 

•  The  schedule  to  deliver  the  ship  has  slipped  from 
September  2015  to  April  2016.  On  September  22,  the  Navy 
announced  that  sea  trials  would  be  delayed  six  to  eight  weeks 
due  to  slower  than  expected  progress  in  the  shipboard  test 
program.  The  development  and  testing  of  EMALS,  AAG, 
DBR,  and  the  Integrated  Warfare  System  will  continue  to  drive 
the  timeline  as  the  ship  progresses  into  test  and  evaluation. 

System 

•  The  CVN  78  Gerald  R.  Ford  class  aircraft  carrier  program  is  a 
new  class  of  nuclear-powered  aircraft  carriers.  It  has  the  same 
hull  form  as  the  CVN  68  Nimitz  class,  but  many  ship  systems, 
including  the  nuclear  plant  and  the  flight  deck,  are  new. 

•  The  newly  designed  nuclear  power  plant  is  intended  to  operate 
at  a  reduced  manning  level  that  is  50  percent  of  a  CVN  68 
class  ship  and  produce  significantly  more  electricity. 

•  The  CVN  78  will  incorporate  EMALS  (electromagnetic, 
instead  of  steam-powered  catapult  launchers)  and  AAG,  and 
will  have  a  smaller  island  with  a  DBR  (phased-array  radars, 
which  replaces/combines  several  legacy  radars  used  on 
current  aircraft  carriers  serving  in  air  traffic  control  and  in  ship 
self-defense). 

•  The  Navy  intends  for  the  Integrated  Warfare  System  to 
be  adaptable  to  technology  upgrades  and  varied  missions 


throughout  the  ship’s  projected  operating  life  including 
increased  self-defense  capabilities  compared  to  current  aircraft 
carriers. 

•  The  Navy  redesigned  weapons  stowage,  handling  spaces,  and 
elevators  to  reduce  manning,  increase  safety,  and  increase 
throughput  of  weapons. 

•  CVN  78  has  design  features  intended  to  enhance  its  ability 
to  launch,  recover,  and  service  aircraft,  such  as  a  slightly 
larger  flight  deck,  dedicated  weapons  handling  areas,  and  an 
increased  number  of  aircraft  refueling  stations.  The  Navy 
set  the  SGR  requirement  for  CVN  78  to  increase  the  sortie 
generation  capability  of  embarked  aircraft  to  160  sorties  per 
day  (12-hour  fly  day)  and  to  surge  to  270  sorties  per  day 
(24-hour  fly  day)  as  compared  to  the  CVN  68  Nimitz  class 
SGR  demonstration  of  120  sorties  per  day/240  sorties  for 
24-hour  surge. 

•  The  Consolidated  Afloat  Networks  and  Enterprise  Service 
(CANES)  program  replaces  five  shipboard  legacy  network 
programs  to  provide  a  common  computing  environment  for 
command,  control,  intelligence,  and  logistics. 

•  CVN  78  is  intended  to  support  the  F-35  and  future  weapons 
systems  over  the  expected  50-year  ship’s  lifespan.  CVN  78 
will  include  a  new  Heavy  UNREP  system  that  will  transfer 
cargo  loads  of  up  to  12,000  pounds. 

•  The  Navy  will  achieve  CVN  78  Initial  Operational  Capability 
in  FY17  after  successful  completion  of  Post  Shakedown 
Availability  and  will  achieve  Full  Operational  Capability  in 
FY19  after  successful  completion  of  IOT&E  testing  and  Type 
Commander  certification. 

Mission 

Carrier  Strike  Group  Commanders  will  use  the  CVN  78  to: 

•  Conduct  power  projection  and  strike  warfare  missions  using 
embarked  aircraft 

•  Provide  force  and  area  protection 

•  Provide  a  sea  base  as  both  a  command  and  control  platform 
and  an  air-capable  unit 

Major  Contractor 

Huntington  Ingalls  Industries,  Newport  News 

Shipbuilding  -  Newport  News,  Virginia 


Activity 

Test  Planning 

•  The  CVN  78  Gerald  R.  Ford  class  carrier  Program  Office 
revised  the  TEMP  1610  to  align  planned  developmental 
tests  with  corresponding  operational  test  phases  and  to 
identify  platform-level  developmental  testing.  DOT&E 
disapproved  this  TEMP  1610  Revision  C  pending  the 
rescheduling  of  the  CVN  78  Class  FSST  from  CVN  79  to 
CVN  78,  before  her  first  operational  deployment. 

CVN  78 


•  The  Navy  is  updating  the  Post  Delivery  Test  and  Trials 
schedule  to  incorporate  the  FSST  as  directed  by  the  Deputy 
Secretary  of  Defense. 

•  The  Navy  plans  a  live  test  to  demonstrate  the  SGR 
with  six  consecutive  12-hour  fly  days  followed  by 

two  consecutive  24-hour  fly  days.  DOT&E  concurs  with 
this  live  test  approach;  however,  resolution  of  how  the  Navy 
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will  extrapolate  the  days  of  live  results  to  the  35-day  design 
reference  mission  on  which  the  SGR  requirement  is  based  is 
yet  to  be  decided.  Until  this  year,  the  Navy  planned  to  use 
a  model  in  development  by  Huntington  Ingalls  Industries 
to  extrapolate  the  live  test  results.  In  June  2015,  COTF 
told  the  CVN  78  program  manager  that  because  several  of 
the  assumptions  tied  to  this  Key  Perfonnance  Parameter 
are  beyond  the  scope  of  operational  test,  COTF  would  not 
accredit  the  Navy’s  Virtual  Carrier  (VCVN)  model  for  use 
during  IOT&E.  DOT&E  agrees  with  COTF's  concerns 
about  the  Key  Performance  Parameter  assumptions,  and  the 
resulting  limitations  of  the  VCVN  model. 

EMALS 

•  The  Navy  is  conducting  installation  and  checkout  of  the 
EMALS  in  CVN  78.  Initial  dead  load  tests  have  been 
completed  on  the  bow  catapults,  and  testing  continues  on 
the  waist  catapults.  To  date,  109  dead  loads  and  191  no 
load  tests  have  been  completed  on  the  bow  catapults,  and 
55  no  load  tests  have  been  completed  on  the  waist  catapults. 

•  The  EMALS  functional  demonstration  test  site  at  Joint 
Base  McGuire-Dix-Lakehurst,  New  Jersey,  continues  to 
test  the  electromagnetic  catapult  system.  The  Navy  has 
also  conducted  over  3,500  dead-load  launches  (non-aircraft, 
weight  equivalent,  and  simulated  launches)  and  over 

450  aircraft  launches  at  the  functional  demonstration  test 
site. 

•  In  2014,  testing  discovered  excessive  EMALS  holdback 
release  dynamics  during  F/A-18E/F  and  EA-18G  catapult 
launches  with  wing-mounted  480-gallon  EFTs.  During  test 
launches,  the  stress  limits  of  the  aircraft  were  exceeded. 

AAG 

•  The  Navy  is  conducting  installation  and  checkout  of  the 
AAG  in  CVN  78.  Hardware  checkout  has  occurred  in 
preparation  for  initial  shipboard  testing. 

•  The  Navy  continues  to  test  the  AAG  on  a  jet  car  track  at 
Joint  Base  McGuire-Dix-Lakehurst,  New  Jersey.  Earlier 
testing  prompted  system  design  changes  that  are  now 
being  tested.  The  jet  car  track  testing  has  examined  the 
F/A-18E/F  performance  envelope  with  the  new  design. 
Overall,  land  based  jet  car  track  testing  has  conducted 

a  total  of  1046  deadload  arrestments;  including,  the 
completion  of  76  performance  deadload  arrestments  in 
4QFY15. 

•  Testing  has  focused  system  performance  of  off  center  and 
angled  (or  skew)  recoveries  that  create  system  instability. 
This  instability  is  known  as  divergent  trajectory  and  is 
created  when  an  aircraft  runout  trajectory  diverges  from  off 
center  and/or  skew  engagement  conditions. 

•  Previously,  the  Navy  de-scoped  the  number  4  AAG  engine, 
reducing  the  total  arresting  gear  engines  on  the  ship, 
including  the  barricade,  to  three,  and  diverted  the  equipment 
to  Runway  Arrested  Landing  Site  in  Lakehurst  to  support 
the  test  program. 

CANES 

•  The  Navy  completed  CANES  integrated  testing  and 
currently  is  performing  follow-on  operational  testing  of 


the  force-level  CANES  configuration  used  on  the  Nimitz 
and  Gerald  R.  Ford  classes.  This  FOT&E  is  scheduled  to 
complete  in  1QFY16. 

•  The  Navy  conducted  integrated  testing  and  IOT&E  of  the 
unit-level  Aegis  destroyer  configuration  in  3QFY14  and 
2QFY15.  The  system  was  operationally  effective,  suitable, 
and  survivable  to  the  cyber  threats  represented  in  the  test. 

DBR 

•  The  radar  consists  of  fixed  array  antennas  both  in  the 

X-  and  S-bands.  The  X-band  radar  is  the  Multi-Function 
Radar  (MFR)  and  the  S-band  radar  is  the  Volume  Search 
Radar. 

•  The  Navy  is  testing  a  production  array  MFR  and  an 
Engineering  Development  Model  array  of  the  Volume 
Search  Radar  at  the  Surface  Combat  System  Center  at 
Wallops  Island,  Virginia.  The  developmental  testing  of 
DBR  resumed  in  4QFY14  at  Wallops  Island  and  is  expected 
to  continue  through  3QFY16.  The  MFR  will  then  be 
installed  on  the  Self-Defense  Test  Ship  for  further  CVN  78 
testing  beginning  2QFY17. 

•  Testing  of  the  production  DBR  has  begun  on  CVN  78  in  the 
shipyard.  Initial  checkout  of  the  equipment  has  occurred. 

Manning 

•  The  Navy  conducted  CVN  78  Manning  War  Game  III  in 
July  2014  to  identify  CVN  78  unique  manpower,  personnel, 
training,  and  education  planning  and  execution  concerns. 

LFT&E 

•  On  August  7,  2015,  the  Deputy  Secretary  of  Defense 
directed  the  Navy  to  complete  the  FSST  before  CVN  78’s 
first  operational  deployment.  The  Revision  A  of  the 
LFT&E  Management  Plan  prepared  by  the  Navy  and 
approved  by  DOT&E  on  July  17,  2007,  stated  the  FSST 
would  be  conducted  on  CVN  78.  The  Navy  unilaterally 
reneged  on  the  approved  strategy  on  June  18,  2012. 

DOT&E  did  not  approve  of  the  Navy  revisions  to  the  new 
Live  Fire  Strategy  and  the  Deputy  Secretary  of  Defense 
concurred  with  DOT&E. 

Assessment 

Test  Planning 

•  A  TEMP  1610  revision  is  under  development  to  address 
problems  with  the  currently-approved  TEMP  1610, 

Revision  B.  The  Navy  submitted  a  revised  TEMP  1610, 
Revision  C  that  was  disapproved  on  February  2,  2015, 
because  the  Navy  removed  the  previously  (2007)  agreed 
upon  FSST.  However,  Revision  C  improved  integrated 
platform-level  developmental  testing,  reducing  the 
likelihood  that  platform-level  problems  will  be  discovered 
during  IOT&E.  In  addition,  the  Program  Office  is  in  the 
process  of  refining  the  post-delivery  schedule  to  further 
integrate  testing.  With  the  Deputy  Secretary  of  Defense’s 
direction  to  the  Navy  to  conduct  the  FSST  before  the 
initial  deployment  on  CVN  78,  the  Navy  desires  to  update 
TEMP  1610,  Revision  C.  DOT&E  has  not  seen  the  Navy’s 
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revision  plan  and  given  the  extent  of  the  updates  required 
may  require  a  Revision  D  to  TEMP  1610. 

•  The  current  state  of  the  Navy’s  VCVN  model  does  not 
fully  provide  for  an  accurate  accounting  of  SGR  due  to  a 
lack  of  fidelity  regarding  manning  and  equipment/aircraft 
availability.  Due  to  these  limitations,  in  June  2015, 

COTF  rescinded  the  use  of  VCVN  for  extrapolating  live 
test  results.  The  Navy  has  not  stated  how  it  intends  to 
extrapolate  the  live  results  to  the  35-day  design  reference 
mission  on  which  the  SGR  requirement  is  based.  DOT&E 
agrees  with  the  COTF  decision.  An  alternative  SGR 
modeling  and  simulation  approach  should  be  developed  by 
the  Navy. 

•  The  schedule  to  deliver  the  ship  has  slipped  from 
September  2015  to  April  2016.  On  September  22,  the 
Navy  announced  that  sea  trials  would  be  delayed  six 
to  eight  weeks  due  to  slower  than  expected  progress  in 
the  shipboard  test  program.  The  ship’s  post-shipyard 
shakedown  availability  will  follow  delivery  in  late 
2016.  During  the  post-shipyard  shakedown  availability, 
installations  of  some  systems  will  be  completed.  The 
first  at-sea  operational  test  and  evaluation  of  CVN  78  is 
scheduled  to  begin  in  September  2017. 

Reliability 

•  CVN  78  includes  several  systems  that  are  new  to  aircraft 
carriers;  four  of  these  systems  stand  out  as  being  critical  to 
flight  operations:  EMALS,  AAG,  DBR,  and  the  Advanced 
Weapons  Elevators  (AWEs).  Overall,  the  uncertain 
reliability  of  these  four  systems  is  the  most  significant  risk 
to  the  CVN-78  IOT&E.  All  four  of  these  systems  are  being 
tested  for  the  first  time  in  their  shipboard  configurations 
aboard  CVN  78.  Reliability  estimates  derived  from  test 
data  for  EMALS  and  AAG  are  discussed  below.  For 
DBR  and  AWE,  reliability  data  collection  has  not  yet 
been  reported  to  DOT&E,  but  is  expected  to  start  at  the 
completion  of  shipboard  installation  and  checkout.  Only 
engineering  reliability  estimates  have  been  provided  to  date. 

•  CVN  78  will  include  a  new  Fleavy  UNREP  system  that  will 
transfer  cargo  loads  of  up  to  12,000  pounds.  Currently, 
only  one  resupply  ship  has  Heavy  UNREP  on  one  station. 
The  Navy  plans  to  install  a  single  Heavy  UNREP  station 
on  each  additional  resupply  ship  beginning  in  FY21  with 
T-AO(X). 

EMALS 

•  EMALS  is  one  of  the  four  systems  critical  to  flight 
operations.  While  testing  to  date  has  demonstrated  that 
EMALS  should  be  able  to  launch  aircraft  planned  for 
CVN  78 ’s  air  wing,  present  limitations  on  F/A-18E/F  and 
EA-18G  configurations,  as  well  as  the  system’s  reliability 
remains  uncertain. 

•  With  the  current  limitations  on  EMALS  for  launching 
the  F/A-18E/F  and  EA-18G  in  operational  configurations 
(i.e., during  test  launches  with  wing-mounted  480-gallon 
EFTs,  the  stress  limits  of  the  aircraft  were  exceeded), 

CVN  78  will  be  able  to  fly  F/A-18E/F  and  EA-18G,  but  not 
in  the  configuration  that  is  required  for  normal  operations. 


If  uncorrected,  this  problem  would  preclude  normal 
employment  from  CVN-78.  Presently,  this  configuration 
substantially  reduces  the  operational  effectiveness  in  of 
F/A-18E/F  and  EA-18G  flying  combat  missions  from  CVN 
78.  The  Navy  has  conducted  deadload  launches  for  changes 
to  the  EMALS  Control  Software  to  correct  this  issue  in 
preparation  for  land  based  aircraft  test  launches  in  3QFY16. 

•  In  FY15,  the  Navy  identified  an  inability  to  readily 
electrically  isolate  EMALS  components  to  perform 
concurrent  maintenance.  For  safety  of  personnel, 
maintenance  and  repair  to  catapults  will  likely  be  limited  to 
non-flight  operations  periods.  It  is  not  possible  to  readily 
electrically  isolate  equipment  during  flight  operations  due 
to  the  shared  nature  of  the  Energy  Storage  Groups  (ESGs) 
and  Power  Conversion  Subsystem  inverters  in  the  four 
launcher/three  ESG  configuration.  The  primary  means 

of  physically  disconnecting  major  subsystems  and  the 
launchers  are  the  Cable  Disconnect  Units  (CDUs).  There 
is  no  circuit  breaker  or  switch  to  secure  power  to  the  CDU; 
CDUs  can  only  be  disconnected  by  first  securing  all  feeding 
power,  dissipating  all  stored  energy  including  spinning 
down  the  motor/generators,  discharging  capacitors,  and 
then  unbolting  and  removing  the  bus  disconnect  links.  This 
provision  would  prevent  certain  maintenance  and  repair 
of  launcher  components  while  power  is  present  in  other 
components  and  while  other  launchers  are  conducting 
flight  operations.  In  contrast,  on  Nimitz  class  carriers  with 
steam  catapults,  maintenance  on  non-operating  catapults 
while  flight  operations  are  performed  on  operating 
catapults  is  allowed  and  routine.  The  effects  on  operational 
performance  of  this  are  unclear,  and  will  depend  upon 
the  extent  to  which  EMALS  redundancy  permits  catapult 
operations  to  continue  not  withstanding  component 
equipment  failures. 

•  In  October  2015,  the  Navy  discovered  that  one  of  three 
PPIS  TRs  had  been  damaged  during  shipboard  certification 
testing.  Two  of  the  three  TRs  are  required  for  normal 
catapult  operations.  The  TRs  were  designed  to  last  the  life 
of  the  ship.  Earlier  faults  discovered  during  developmental 
testing  resulted  in  stepwise  improvements  to  the  PPIS  TR 
design  and  construction.  This  failed  TR  had  one  of  the  four 
improvements.  The  PPIS  is  130  inches  wide,  74  inches 
deep,  80  inches  high,  and  weighs  over  35,000  pounds.  The 
replacement  PPIS  will  be  shipped  to  and  fault  checked 

at  Joint  Base  McGuire-Dix-Lakehurst,  New  Jersey,  and 
then  shipped  to  Newport  News,  Virginia,  for  installation 
on  CVN  78.  The  removal  of  the  old  PPIS,  which,  due  to 
the  size  and  mass  of  the  PPIS  will  require  cutting  a  hole 
in  the  ship’s  hull,  and  installation  of  the  new  one  will  take 
several  months,  but  is  not  expected  to  delay  testing  or  ship’s 
delivery. 

•  As  of  December  2014,  the  program  estimates  that  EMALS 
has  approximately  340  Mean  Cycles  Between  Critical 
Failure  (MCBCF)  in  the  shipboard  configuration,  where 

a  cycle  represents  the  launch  of  one  aircraft.  While  this 
estimate  is  above  the  re-baselined  reliability  growth  curve, 
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the  re -baselined  curve  is  well  below  the  requirement 
of  4,166  MCBCF.  The  failure  rate  for  the  last  reported 
MCBCF  was  3.7  times  higher  than  should  have  been 
expected  at  this  point  in  the  development.  Absent  a  major 
redesign,  it  is  unlikely  EMALS  will  be  capable  of  meeting 
the  requirement  of  4,166  MCBCF. 

AAG 

•  AAG  is  another  system  critical  to  flight  operations. 

Testing  to  date  has  demonstrated  that  AAG  should  be 
able  to  recover  aircraft  planned  for  the  CVN  78  air  wing, 
but  AAG’s  reliability  is  uncertain.  The  Program  Office 
redesigned  major  components  that  did  not  meet  system 
specifications  during  land-based  testing.  The  Program 
Office  last  provided  reliability  data  in  December  2013  and 
estimated  that  AAG  had  approximately  20  Mean  Cycles 
Between  Operational  Mission  Failure  (MCBOMF)  in 

the  shipboard  configuration,  where  a  cycle  represents  the 
recovery  of  one  aircraft.  The  requirement  is  an  MCBOMF 
of  16,500.  The  Program  Office  expects  to  have  a  reliability 
estimate  for  the  new  design  by  the  end  of  2015.  The  last 
reported  failure  rate  was  248  times  higher  than  should  have 
been  expected  at  this  point  in  the  development. 

DBR 

•  Previous  testing  of  Navy  combat  systems  similar  to 
CVN  78 ’s  revealed  numerous  integration  problems  that 
degrade  the  performance  of  the  combat  system.  Many  of 
these  problems  are  expected  to  exist  on  CVN  78.  The  DBR 
testing  at  Wallops  Island  is  typical  of  early  developmental 
testing  with  the  system  still  in  the  problem  discovery 
phase.  Current  results  reveal  problems  with  tracking 

and  supporting  missiles  in  flight,  excessive  numbers  of 
clutter/false  tracks,  and  track  continuity  concerns.  More 
test-analyze-fix  cycles  are  necessary  for  DBR  to  develop 
and  test  fixes  so  that  it  can  properly  perform  air  traffic 
control  and  engagement  support  on  CVN  78.  Previous  test 
results  emphasize  the  necessity  of  maintaining  a  DBR/CVN 
78  combat  system  asset  at  Wallops  Island.  The  removal  of 
the  MFR  and  the  conclusion  of  developmental  testing  was 
originally  scheduled  for  3QFY15,  but  the  Navy  decided 
to  extend  the  Wallops  Island  testing  through  3QFY16. 
DOT&E  concurs  with  this  schedule  change  and  considers 
it  a  necessary  part  of  delivering  a  fully-capable  combat 
system  in  CVN  78. 

SGR 

•  It  is  unlikely  that  CVN  78  will  achieve  its  SGR 
requirement.  The  target  threshold  is  based  on  unrealistic 
assumptions  including  fair  weather  and  unlimited  visibility, 
and  that  aircraft  emergencies,  failures  of  shipboard 
equipment,  ship  maneuvers,  and  manning  shortfalls  will  not 
affect  flight  operations.  DOT&E  plans  to  assess  CVN  78 
performance  during  IOT&E  by  comparing  it  to  the  SGR 
requirement  as  well  as  to  the  demonstrated  performance  of 
the  Nimitz  class  carriers. 

•  During  the  2013  operational  assessment,  DOT&E 
conducted  an  analysis  of  past  aircraft  carrier  operations  in 
major  conflicts.  The  analysis  concludes  that  the  CVN  78 


SGR  requirement  is  well  above  historical  levels  and  that 
CVN  78  is  unlikely  to  achieve  that  requirement.  There  are 
concerns  with  the  reliability  of  key  systems  that  support 
sortie  generation  on  CVN  78.  Poor  reliability  of  these 
critical  systems  could  cause  a  cascading  series  of  delays 
during  flight  operations  that  would  affect  CVN  78 ’s 
ability  to  generate  sorties,  make  the  ship  more  vulnerable 
to  attack,  or  create  limitations  during  routine  operations. 
DOT&E  assesses  the  poor  or  unknown  reliability  of  these 
critical  subsystems  will  be  the  most  significant  risk  to 
CVN  78’s  successful  completion  oflOT&E.  The  analysis 
also  considered  the  operational  implications  of  a  shortfall 
and  concluded  that  as  long  as  CVN  78  is  able  to  generate 
sorties  comparable  to  Nimitz  class  carriers,  the  operational 
implications  of  CVN  78  will  be  similar  to  that  of  a  Nimitz 
class  carrier. 

Manning 

•  The  latest  Navy  analysis  of  manning  identified  several  areas 
of  concern.  The  Navy  has  re-designated  some  officer  rooms 
as  Chief  Petty  Officer  (CPO)  berthing  spaces  to  resolve  a 
shortfall  in  CPO  berthing. 

•  During  some  exercises,  the  berthing  capacity  for  officers 
and  enlisted  will  be  exceeded,  requiring  the  number  of 
evaluators  to  be  limited  or  the  timeframe  to  conduct  the 
training  to  be  lengthened.  This  shortfall  in  berthing  is 
further  exacerbated  by  the  246  officer  and  enlisted  billets 
(roughly  10  percent  of  the  crew)  identified  in  the  Manning 
War  Game  III  as  requiring  a  face-to-face  turnover.  These 
turnovers  will  not  all  happen  at  one  time,  but  will  require 
heavy  oversight  and  will  limit  the  amount  of  turnover  that 
can  be  accomplished  at  sea  and  especially  during  evaluation 
periods. 

•  Manning  must  be  supported  at  the  100  percent  level, 
although  this  is  not  the  Navy’s  standard  practice  on  other 
ships  and  the  Navy’s  personnel  and  training  systems  may 
not  be  able  to  support  100  percent  manning.  The  ship  is 
extremely  sensitive  to  manpower  fluctuations.  Workload 
estimates  for  the  many  new  technologies  such  as  catapults, 
arresting  gear,  radar,  and  weapons  and  aircraft  elevators  are 
not  yet  well-understood.  Finally,  the  Navy  is  considering 
placing  the  ship’s  seven  computer  networks  under  a  single 
department.  Network  management  and  the  correct  manning 
to  facilitate  continued  operations  is  a  concern  for  a  network 
that  is  more  complex  than  historically  seen  on  Navy  ships. 

LFT&E 

•  The  Navy  has  made  substantial  progress  on  defining 
the  scope  of  the  Total  Ship  Survivability  Trial  and  the 
Analytical  Bridge  task.  While  these  portions  of  the  LFT&E 
Management  Plan  were  adequately  defined  in  the  Revision 
B  document,  DOT&E  returned  the  LFT&E  Management 
Plan  to  the  Navy  solely  on  the  basis  of  the  FSST  on 

CVN  79  verses  CVN  78.  With  the  Deputy  Secretary  of 
Defense’s  direction  to  the  Navy  to  reinsert  the  FSST,  a 
revised  LFT&E  Management  Plan  is  under  development. 

•  CVN  78  has  many  new  critical  systems,  such  as  EMALS, 
AAG,  AWE,  and  DBR  that  have  not  undergone  shock  trials 
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on  other  platforms.  Unlike  past  tests  on  other  new  classes 
of  ships  with  legacy  systems,  the  performance  of  CVN-78’s 
new  critical  systems  is  unknown.  Inclusion  of  data  from 
shock  trials  early  in  a  program  has  been  an  essential 
component  of  building  survivable  ships.  The  current  state 
of  modeling  and  component-level  testing  are  not  adequate 
to  identify  the  myriad  of  problems  that  have  been  revealed 
only  through  full  ship  shock  testing. 

•  The  FSST  and  component  shock  qualification  test  data 
could  affect  the  design  of  future  carriers  in  the  class  and 
are  critical  to  the  assessment  of  the  CVN  78  survivability 
against  operationally  relevant  threats.  The  FSST  is 
scheduled  to  occur  on  CVN  78  in  FY19. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  should 
continue  to  address  the  seven  remaining  FY10,  FY11,  FY13, 
and  FY14  recommendations. 

1.  Finalize  plans  that  address  CVN  78  Integrated  Warfare 
System  engineering  and  ship’s  self-defense  system 
discrepancies  prior  to  the  start  of  IOT&E. 

2.  Continue  aggressive  EMALS  and  AAG  risk-reduction 
efforts  to  maximize  opportunity  for  successful  system 
design  and  test  completion  in  time  to  meet  required  in-yard 
dates  for  shipboard  installation  of  components. 

3.  Provide  scheduling,  funding,  and  execution  plans  to 
DOT&E  for  the  live  SGR  test  event  during  the  IOT&E. 


4.  Continue  to  work  with  the  Navy’s  Bureau  of  Personnel  to 
achieve  adequate  depth  and  breadth  of  required  personnel 
to  sufficiently  meet  Navy  Enlisted  Classification  fit/fill 
manning  requirements  of  CVN  78. 

5.  Conduct  system-o f-systems  developmental  testing  to 
preclude  discovery  of  deficiencies  during  IOT&E. 

6.  Address  the  uncertain  reliability  of  EMALS,  AAG,  DBR, 
and  AWE.  These  systems  are  critical  to  CVN  78  flight 
operations,  and  are  the  largest  risk  to  the  program. 

7.  Aggressively  fund  and  address  a  solution  for  the  excessive 
EMALS  holdback  release  dynamics  during  F/A-18E/F  and 
EA-18G  catapult  launches  with  wing-mounted  480-gallon 
EFTs. 

•  FY15  Recommendations.  The  Navy  should: 

1 .  Ensure  the  continuation  of  funding  and  testing  of  the  DBR 
at  Wallops  Island  through  3QYFY16  address  the  problems 
discovered  during  initial  developmental  testing. 

2.  Begin  tracking  and  reporting  on  a  quarterly  basis  systems 
reliability  for  all  new  systems  but  at  a  minimum  for 
EMALS,  AAG,  DBR,  and  AWE. 

3.  The  Navy  should  ensure  the  continued  funding  for 
component  shock  qualification  of  both  government  and 
contractor  furnished  equipment. 

4.  Submit  a  TEMP  for  review  and  approval  by  DOT&E 
incorporating  the  Deputy  Secretary’s  direction  to  conduct 
the  FSST  before  CVN  78’s  first  deployment. 
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DDG  1000  Zumwalt  Class  Destroyer 


Executive  Summary 

•  The  first  ship  in  the  DDG  1000  Zumwalt  class  destroyers 
was  launched  on  October  28,  2013.  The  Navy  initiated 
pre -deli very  testing  in  FY13  on  DDG  1000  (lead  ship),  and 
testing  will  continue  in  FY16.  The  Navy  plans  to  accept 
delivery  of  DDG  1000  in  an  incomplete  condition  and  sail 
the  ship  to  the  West  Coast  in  FY16.  After  the  ship  arrives 
on  the  West  Coast,  it  will  begin  an  18-month  post-delivery 
availability  to  complete  installation,  integration,  and  shipyard 
testing  of  mission  systems.  The  Navy  plans  to  conduct  a 
second  Acceptance  Trial  when  that  availability  has  been 
completed  and  expects  IOT&E  to  commence  in  FY18. 

•  The  Navy  removed  funding  for  the  planned  Full  Ship  Shock 
Trial  (FSST)  on  DDG  1000  in  September  2014,  unilaterally 
deciding  to  conduct  the  event  on  DDG  1002.  In  October  2015, 
the  Navy  revised  their  decision  and  agreed  to  conduct  FSST 
(specific  ship  to  be  determined)  prior  to  the  first  deployment  of 
any  DDG  1000  Zumwalt  class  destroyer. 

•  To  complete  the  survivability  analysis,  the  Navy  needs 
to  restore  funding  and  complete  the  component  shock 
qualification  program  that  is  used  to  verify  if  the  equipment 
installed  in  all  three  hulls  meets  the  design  requirements.  The 
Navy  also  needs  to  address  the  shortfalls  associated  with  the 
models  that  support  the  overall  survivability  evaluation. 

System 

The  DDG  1000  Zumwalt  class  destroyers  are  new  surface 
combatants  with  a  wave-piercing  tumblehome  hull  form  designed 
both  for  endurance  and  low-radar  detectability.  The  Navy 
currently  plans  to  acquire  three  ships  of  the  class.  The  DDG 
1000  Zumwalt  class  destroyer  is  equipped  with  the  following: 

•  Total  Ship  Computing  Environment  Infrastructure  that  hosts 
all  ship  functions  on  an  integrated,  distributed  computing 
plant. 

•  Two  155  mm  Advanced  Gun  Systems  that  fire  Long  Range 
Land  Attack  Projectiles  (LRLAPs). 

•  AN/SPY-3  Multi-Function  (X-band)  radar  modified  to  include 
a  volume  search  capability.  (The  Navy  removed  the  Volume 
Search  Radar  (S-band)  from  the  ship’s  baseline  design  for 
cost  reduction  in  compliance  with  an  Acquisition  Decision 
Memorandum  of  June  1,  2010.) 

•  Eighty  vertical  launch  cells  that  can  hold  a  mix  of  Tomahawk 
Land  Attack  Missiles,  Standard  Missiles,  Vertical  Launch 
Anti-Submarine  Rockets,  and  Evolved  Sea  Sparrow  Missiles. 


•  An  integrated  Undersea  Warfare  system  with  a  dual  frequency 
bow-mounted  sonar  and  multi-function  towed  array  sonar  to 
detect  submarines  and  assist  in  avoiding  mines. 

•  An  ability  to  embark  and  maintain  MH-60R  helicopters  and 
vertical  take-off  unmanned  aerial  vehicles. 

•  An  Integrated  Power  System  that  can  direct  electrical  power  to 
propulsion  motors,  combat  systems,  or  other  ship  needs. 

Mission 

•  The  Joint  Force  Maritime  Component  Commander  intends  to 
employ  the  DDG  1000  Zumwalt  class  destroyer  to  provide: 

-  Joint  Surface  Strike  /Power  Projection 

-  Joint  Surface  Fire  Support 

-  Surface  Warfare 

-  Anti-Air  Warfare 

-  Anti-Submarine  Warfare 

•  The  DDG  1000  Zumwalt  class  destroyer  is  intended  to  operate 
independently  or  in  conjunction  with  an  Expeditionary  or 
Carrier  Strike  Group,  as  well  as  with  other  joint  or  coalition 
partners  in  a  Combined  Expeditionary  Force  environment. 

Major  Contractors 

•  General  Dynamics  Marine  Systems  Bath  Iron  Works  -  Bath, 
Maine 

•  Huntington  Ingalls  Industries  -  Pascagoula,  Mississippi 

•  BAE  Systems  -  Minneapolis,  Minnesota 

•  Raytheon  -  Waltham,  Massachusetts 


Activity 

•  The  Navy  continues  to  revise  the  Test  and  Evaluation  Master 
Plan  (TEMP).  The  most  significant  changes  being  addressed 
in  the  TEMP  revision  are: 


Modification  of  the  AN/SPY-3  Multi-Function  Radar 
(X  Band,  horizon  search  radar)  to  provide  the  volume 
search  capability  that  would  have  been  provided  by  the 
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Volume  Search  Radar  (S-band),  which  has  been  removed 
from  the  baseline  design. 

-  Replacement  of  two  MK  110  57  mm  close-in  gun  systems 
(integrated  with  the  ship’s  combat  system),  with  two 
standalone  MK  46  30  mm  guns  (not  integrated  with  the 
ship’s  combat  system). 

-  Incorporation  of  schedule  adjustments  that  reflect  shipyard 
delays  and  the  Navy’s  plan  to  complete  component 
installation,  integration,  and  mission  system  testing  on 
DDG  1000  during  an  18-month  availability  at  a  West  Coast 
shipyard.  The  Navy  plans  to  conduct  a  second  Acceptance 
Trial  when  that  availability  has  been  completed  and  expects 
IOT&E  to  commence  in  FY18. 

-  Inclusion  of  detailed  test  designs  for  major  mission  areas, 
including  Anti-Air  Warfare,  Anti-Submarine  Warfare, 
Surface  Warfare,  fire  support,  and  mine  avoidance. 

•  The  Navy  completed  LRLAP  lethality  testing  in  FY14. 
Modeling  and  simulation  will  be  used  to  assess  LRLAP 
performance  against  multiple  targets  in  2017. 

•  The  Navy  continues  development  of  the  DDG  1000 
Probability  of  Raid  Annihilation  test  bed,  which  is  modeling 
and  simulation  that  will  be  used,  in  conjunction  with  live  fire 
testing  using  the  Self-Defense  Test  Ship,  to  assess  DDG  1000’s 
capability  to  defeat  threat  anti-ship  cruise  missiles  and  aircraft. 

•  In  January  2013,  DOT&E  sent  a  memorandum  to  the 
Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  outlining  the  need  for  a  threat  torpedo  surrogate 
to  support  operational  testing  of  DDG  1000  and  other 
ships/ submarines,  and  requesting  the  Navy’s  plan  to  address 
this  need. 

-  In  June  2015,  DOT&E  sent  a  follow-up  memorandum 
that  reiterated  the  need  for  adequate  torpedo  surrogates  in 
operational  test  and  noted  that  DOT&E  has  yet  to  receive 
the  Navy’s  plan. 

-  In  September  2015,  the  Navy  completed  a  formal  study  to 
identify  capability  gaps  in  the  currently  available  torpedo 
surrogates  and  to  present  an  analysis  of  alternatives  for 
specific  investments  to  improve  threat  emulation  ability. 

•  In  September  2014,  the  Navy  requested  DOT&E  concurrence 
to  move  FSST  from  the  first  ship  of  its  class,  DDG  1000, 

to  the  last  of  its  class,  DDG  1002.  DOT&E  disapproved 
the  deferral  request  and  opposed  changing  the  schedule  and 
test  article  resourcing  within  a  TEMP  revision.  The  Navy 
unilaterally  removed  the  funding  for  the  FSST  and  proceeded 
with  their  unfunded  plan  to  conduct  FSST  on  DDG  1002.  In 
October  2015,  the  Navy  revised  their  decision  and  agreed  to 
conduct  FSST  (specific  ship  to  be  determined)  prior  to  the 
first  deployment  of  any  DDG  1000  Zumwalt  class  destroyer. 
However,  the  component  shock  qualification  program  for 
DDG  1000  Zumwalt  remains  incomplete  and  is  unfunded  for 
completion. 

•  In  April  2015,  the  Navy  notified  DOT&E  that  the  underwater 
explosion  (UNDEX)  vulnerability  assessments  would  be 


delayed  due  to  problems  with  the  full  ship  finite  element 
analysis  model.  In  addition,  DOT&E  was  notified  that  the 
recently  developed  blast  module  for  the  air  explosion  (AIREX) 
vulnerability  assessment  of  the  as-built  configuration  of  the 
ship  cannot  be  operated  within  the  Advance  Survivability 
Assessment  Program  (ASAP)  simulation  due  to  integration 
problems  with  the  two  sets  of  computer  code.  The  Navy 
intends  to  use  available  model  and  simulation  tools  and 
resultant  vulnerability  predictions  that  it  previously  concluded 
have  significant  limitations,  as  cited  in  a  2005  Verification 
and  Validation  Assessment  Report  of  ASAP.  The  Navy  report 
identified  several  measures  that  cannot  be  determined  at  the 
necessary  accuracy  or  confidence  using  ASAP. 

Assessment 

•  The  Navy  study  on  threat  torpedo  surrogates  confirmed 
DOT&E’s  concerns  that  current  torpedo  surrogates  have 
significant  gaps  in  threat  representation  for  operational  testing 
and  the  study  provided  recommendations  for  improving 
current  threat  torpedo  emulation.  However,  the  Navy  has  yet 
to  provide  its  plan  to  obtain  adequate  torpedo  surrogates  to 
effectively  characterize  DDG  1000  Zumwalt  class  destroyer 
performance  in  operational  test. 

•  Conducting  FSST  on  DDG  1000  is  critical  to  finding  and 
correcting  failures  in  mission-critical  capabilities  prior  to 
the  classes  first  deployment  and  prior  to  placing  this  class  of 
ships  in  harm’s  way.  FSSTs  routinely  uncover  mission-critical 
vulnerabilities  that  were  not  identified  by  component  testing, 
analysis,  and/or  modeling  and  simulation  alone. 

•  A  component  shock  qualification  program  for  assessing  ship 
vulnerability  to  below-water  threats  is  necessary  for  accurate 
damage  simulations.  However,  the  shock  qualification 
program  remains  unfunded. 

•  All  three  ships  of  the  DDG  1000  Zumwalt  class  have  in 
common  a  significant  amount  of  new  designs,  including 
the  unique  wave-piercing  tumblehome  hull  form,  as  well  as 
the  new  Integrated  Power  System,  Total  Ship  Computing 
Environment  (software,  equipment  and  infrastructure), 
Integrated  Undersea  Warfare  System,  Peripheral  Vertical 
Launching  System,  the  Advanced  Gun  System,  and  the 
associated  automated  magazines.  These  systems  and 
equipment  have  not  been  subjected  to  shock  on  previous 
ship  classes.  Moreover,  the  previously  untried  automation 
and  small  crew  for  a  ship  this  size,  limit  the  sailors’  ability 
to  conduct  repairs  to  enable  recovery  from  shock-induced 
damage. 

•  UNDEX  and  AIREX  vulnerability  assessments  currently 
lack  credible  damage  prediction  models.  The  challenges  and 
limitations  in  predicting  ship  vulnerability  reinforce  the  need 
to  complete  FSST  and  component  shock  qualification  before 
the  first  operational  deployment  of  a  DDG  1000  Zumwalt  class 
destroyer.  DOT&E  will  provide  additional  recommendations 
to  mitigate  or  limit  the  unknown  vulnerability  of  the 
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DDG  1000  Zumwalt  class  destroyer  after  completing  a  more 
detailed  evaluation  of  the  available  model  and  simulation  tools 
and  their  limitations. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  should 
address  the  following  open  recommendations  from  FY13  and 
earlier: 

1 .  Develop  tactics  and  training  that  optimize  employment  of 
the  MK  46  gun  systems  against  surface  threats. 

2.  Fund  and  schedule  component  shock  qualification  to 
support  the  DDG  1000  Zumwalt  class  destroyer  requirement 
to  maintain  all  mission  essential  functions  when  exposed  to 
UNDEX  shock  loading. 

3.  Determine  a  development  and  test  strategy  that  mitigates 
the  risk  of  delivering  substantial  mission  capability  after 
ship  delivery  and  transit  to  the  West  Coast. 


4.  Develop  a  strategy  to  validate  reliability  of  the 
accelerometers  used  in  LRLAP  prior  to  shipboard 
operational  test. 

5.  Develop  and  conduct  an  accreditation  plan  that  validates  the 
acceptability  of  the  Probability  of  Raid  Annihilation  test  bed 
to  support  operational  test. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Fund  and  schedule  FSST  prior  to  the  first  deployment  of 
any  DDG  1000  Zumwalt  class  destroyer  and  formalize 
this  plan  within  revisions  of  the  TEMP  and  LFT&E 
Management  Plan. 

2.  Complete  the  revision  to  the  TEMP  that  accounts  for 
DDG  1000  baseline  changes  and  system  delivery  schedule. 

3.  Develop  torpedo  surrogate(s)  that  can  be  used  to 
characterize  DDG  1000  Zumwalt  class  destroyer  capability 
against  threat  torpedoes  during  operational  test. 
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DDG  51  Flight  III  Destroyer/Air  and  Missile  Defense 
Radar  (AMDR)/Aegis  Combat  System 


Executive  Summary 

•  On  December  10,  2014,  the  Deputy  Secretary  of  Defense 
(DEPSECDEF)  directed  the  Director,  Cost  Analysis/Program 
Evaluation  (CAPE)  to  identify  viable  at-sea  operational 
testing  options  that  meet  DOT&E  adequacy  requirements 
and  recommend  a  course  of  action  (with  cost  estimates, 
risks,  and  benefits)  to  satisfy  testing  of  the  Air  and  Missile 
Defense  Radar  (AMDR),  Aegis  Combat  System,  and  Evolved 
Seasparrow  Missile  (ESSM)  Block  2  in  support  of  the 
DDG  5 1  Flight  III  Destroyer  program. 

•  The  CAPE  study  evaluated  four  options  to  deliver  an  at-sea 
test  platform  adequate  for  self-defense  operational  testing  of 
the  DDG  51  Flight  III,  AMDR,  and  ESSM  Block  2  programs. 
Each  option  requires  funding  beginning  no  later  than  F  Y 1 8  to 
ensure  support  of  operational  testing  of  these  systems  in  FY22. 

•  A  decision  on  whether  to  fund  the  advance  procurement  of 
the  equipment  needed  to  support  the  self-defense  operational 
testing  is  pending. 

System 

•  The  DDG  5 1  Flight  III  Destroyer  will  be  a  combatant  ship 
equipped  with  the: 

-  AMDR  three-dimensional  (range,  altitude,  and  azimuth) 
multi-function  radar 

-  Aegis  Combat  System 

-  AN/SQQ-89  Undersea  Warfare  suite  that  includes  the 
AN/SQS-53  sonar 

-  MH-60R  helicopter 

-  Close-In  Weapon  System 

-  5-inch  diameter  gun 

-  Vertical  Launch  System  that  can  launch  Tomahawk, 
Standard  Missiles-2,  3,  and  6,  and  the  ESSM  Blocks  1 
and  2 

•  The  Navy  is  developing  the  AMDR  to  provide  simultaneous 
sensor  support  of  integrated  air  and  missile  defense  (IAMD) 
and  air  defense  (including  self-defense)  missions.  IAMD  and 
air  defense  missions  require  extended  detection  ranges  and 
increased  radar  sensitivity  against  advanced  threats  with  high 
speeds  and  long  interceptor  fly-out  times.  The  three  major 
components  of  AMDR  are: 

-  The  AMDR  S-band  radar  that  will  provide  IAMD, 
search,  track,  cueing,  missile  discrimination,  air 
defense  non-cooperative  target  recognition,  S-band 
missile  communications,  surveillance  capability  for 
ship  self-defense  and  area  air  defense,  and  S-band  kill 
assessment  support  functions. 

-  The  AMDR  X-band  radar  will  provide  horizon  and 
surface  search  capabilities  and  navigation  and  periscope 
detection/discrimination  functions.  The  Navy  is  delaying 


development  of  the  X-band  radar;  the  existing  AN/SPQ-9B 
radar  will  provide  these  functions  in  the  interim. 

-  The  AMDR  Radar  Suite  Controller  that  will  provide 
radar  resource  management  and  coordination  and  an  open 
interface  with  the  ship’s  combat  system. 

•  The  Aegis  Combat  System  is  an  integrated  naval  weapons 
system  that  uses  computers  and  radars  to  form  an  advanced 
command  and  decision  capability  and  a  weapons  control 
system  to  track  and  guide  weapons  to  destroy  enemy  targets. 

•  The  ESSM,  cooperatively  developed  among  13  nations,  is 
a  medium-range,  ship-launched  self-defense  guided  missile 
designed  to  defeat  ASCM,  surface,  and  low-velocity  air 
threats.  There  are  two  variants  of  ESSM. 

-  ESSM  Block  1  is  a  semi-active  radar-guided  missile  that  is 
currently  in-service. 

-  ESSM  Block  2  is  in  development  and  will  have 
semi-active  radar-guidance  as  well  as  active  radar 
guidance. 

•  In  comparison  to  the  previous  DDG  51  version  (Flight  IIA), 
Flight  III  includes,  in  addition  to  the  Aegis  Combat  System 
and  the  AMDR,  the  following  modifications: 

-  An  upgraded  fire  extinguishing  system 

-  New  ship  service  turbine  generators 

-  Additional  transformers 

-  Power  Conversion  Modules 

-  Modified  controllers  for  the  Machinery  Control  System 
and  Multifunction  Monitors 

-  Upgraded  air-conditioning  plants 

•  Flight  III  is  also  structurally  different  from  the  prior  DDG 
51  version.  The  design  will  add  starboard  enclosures  and  a 
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stack  of  small  boats,  and  additional  structure  in  the  fantail  to 
increase  reserve  buoyancy  and  help  compensate  for  additional 
weight  increase.  It  will  also  include  structural  modifications  to 
increase  plate  thicknesses  to  lower  the  ship’s  center-of-gravity 
and  enhance  girder  strength. 

Mission 

•  The  Navy  will  use  the  DDG  5 1  Flight  III  Destroyer  equipped 
with  the  Aegis  Combat  System  and  AMDR  to  provide  joint 
battlespace  threat  awareness  and  defense  capability  to  counter 
current  and  future  threats  in  support  of  joint  forces  ashore  and 
afloat. 

•  The  Navy  will  use  the  AMDR  S-band  radar/Radar  Suite 
Controller  with  the  AN/SPQ-9B  and  the  Aegis  Combat 
System  to  support  the  following  DDG  5 1  Flight  III  Destroyer 
missions: 

-  Area  air  defense  (to  include  self-defense  with  the  ESSM) 
to  counter  advanced  air  and  cruise  missile  threats  and 
increase  ship  survivability 

-  Detect,  track,  discriminate,  and  provide  missile 
engagement  support  (including  kill  assessment)  to  counter 
ballistic  missile  threats 


-  Counter  surface  threats  through  surface  surveillance, 
precision  tracking,  and  missile  and  gun  engagements 

-  Conduct  Undersea  Warfare  with  periscope  detection  and 
discrimination 

-  Detect  and  track  enemy  artillery  projectiles  to  support 
combat  system  localization  of  land-battery  launch  positions 
by  the  DDG  51  Flight  III  Combat  System 

-  Detect  and  track  own-ship  gun  projectiles  to  support 
surface  warfare  and  naval  surface  fire  support 

Major  Contractors 

•  DDG  5 1  Flight  III  Destroyer:  To  be  determined.  Current 

DDG  51  Destroyer  major  contractors  are: 

-  General  Dynamics  Marine  Systems  Bath  Iron 
Works  -  Bath,  Maine 

-  Huntington  Ingalls  Industries,  Ingalls  Shipbuilding 
Division  -  Pascagoula,  Mississippi 

•  AMDR:  Raytheon  -  Sudbury,  Massachusetts 

•  Aegis  Combat  System:  Lockheed  Martin  Marine  Systems  and 

Sensors  -  Moorestown,  New  Jersey 

•  ESSM  Block  2:  Raytheon  -  Tucson,  Arizona 


Activity 

•  On  December  10,  2014,  DEPSECDEF  issued  a  memorandum 
directing  CAPE  to  identify  viable  at-sea  operational  testing 
options  that  meet  DOT&E  adequacy  requirements  and 
recommend  a  course  of  action  (with  cost  estimates,  risks, 
and  benefits)  to  satisfy  testing  of  the  AMDR,  Aegis  Combat 
System,  and  ESSM  Block  2  programs  in  support  of  the  DDG 
51  Flight  III  Destroyer  program. 

•  The  CAPE  study  evaluated  four  options  to  deliver  an  at-sea 
test  platform  adequate  for  self-defense  operational  testing  of 
these  systems.  Each  option  requires  funding  beginning  in 
FY18  to  ensure  support  of  the  DDG  51  Flight  III,  AMDR,  and 
ESSM  Block  2  operational  testing  in  2022. 

•  In  September  2015,  DOT&E  met  with  DDG  51  Program 
Office  representatives  to  start  planning  the  Flight  III  LFT&E 
program. 

•  DEPSECDEF  has  not  made  a  decision  yet  on  the  funding 
needed. 

Assessment 

•  DOT&E’s  position  continues  to  be  that  the  Navy’s  operational 
test  programs  for  the  AMDR,  Aegis  Combat  System,  ESSM 
Block  2,  and  DDG  51  Flight  III  Destroyer  programs  are  not 
adequate  to  fully  assess  their  self-defense  capabilities.  They 
are  also  not  adequate  to  test  the  following  Navy-approved 
DDG  51  Flight  III,  AMDR,  Aegis  Combat  System,  and  ESSM 
Block  2  requirements. 

-  The  AMDR  Capability  Development  Document  (CDD) 
describes  AMDR’s  IAMD  mission,  which  requires  AMDR 
to  support  simultaneous  defense  against  multiple  ballistic 
missile  threats  and  multiple  advanced  anti-ship  cruise 
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missile  (ASCM)  threats.  The  CDD  also  includes  an 
AMDR  minimum  track  range  Key  Performance  Parameter. 

-  The  DDG  51  Flight  III  Destroyer  has  a  survivability 
Key  Performance  Parameter  requirement  directly  tied 
to  meeting  a  self-defense  requirement  threshold  against 
ASCMs  described  in  the  Navy’s  Surface  Ship  Theater  Air 
and  Missile  Defense  Assessment  document  of  July  2008. 

It  clearly  states  that  area  defense  will  not  defeat  all  the 
threats,  thereby  demonstrating  that  area  air  defense  will 
not  completely  attrite  all  ASCM  raids  and  individual 
ships  must  be  capable  of  defeating  ASCM  leakers  in  the 
self-defense  zone. 

-  The  ESSM  Block  2  CDD  has  a  requirement  to  provide 
self-defense  against  incoming  ASCM  threats  in  clear 
and  jamming  environments.  The  CDD  also  includes  an 
ESSM  Block  2  minimum  intercept  range  Key  Performance 
Parameter. 

•  Use  of  manned  ships  for  operational  testing  with  threat 
representative  ASCM  surrogates  in  the  close-in,  self-defense 
battlespace  is  not  possible  due  to  Navy  safety  restrictions 
because  targets  and  debris  from  intercepts  pose  an 
unacceptable  risk  to  personnel  at  ranges  where  some  of  the 
engagements  will  take  place.  The  November  2013  mishap 
on  the  USS  Chancellorsville  (CG  62)  involving  an  ASCM 
surrogate  target  resulted  in  even  more  stringent  safety 
constraints. 

-  In  addition  to  stand-off  ranges  (on  the  order  of  1.5  to 

5  nautical  miles  for  subsonic  and  supersonic  surrogates, 
respectively),  safety  restrictions  require  that  ASCM 
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targets  not  be  flown  directly  at  a  manned  ship,  but  at 
some  cross-range  offset,  which  unacceptably  degrades  the 
operational  realism  of  the  test. 

-  Similar  range  safety  restrictions  will  preclude  manned  ship 
testing  of  eight  of  the  nine  ASCM  scenarios  contained  in 
the  Navy-approved  requirements  document  for  the  Aegis 
Modernization  Advanced  Capability  Build  16  Combat 
System  upgrade.  Restrictions  also  preclude  testing  of  the 
AMDR  minimum  track  range  requirement  against  threat 
representative  ASCM  threat  surrogates  at  the  land-based 
AMDR  Pacific  Missile  Range  Facility  test  site. 

-  To  overcome  these  safety  restrictions  for  the  LHA  6, 
Littoral  Combat  Ship,  DDG  1000,  LPD  17,  LSD  41/49, 
and  CVN  78  ship  classes,  the  Navy  developed  an  Air 
Warfare/Ship  Self-Defense  Enterprise  Modeling  and 
Simulation  (M&S)  test  bed,  which  uses  live  testing  in 
the  close-in  battlespace  with  targets  flying  realistic  threat 
profiles  and  manned  ship  testing  for  other  battlespace 
regions,  as  well  as  soft-kill  capabilities  to  validate  and 
accredit  the  M&S  test  bed.  The  same  needs  to  be  done 
for  the  DDG  5 1  Flight  III  Destroyer  with  its  AMDR,  as 
side-by-side  comparison  between  credible  live  fire  test 
results  and  M&S  test  results  form  the  basis  for  the  M&S 
accreditation.  Without  a  Self-Defense  Test  Ship  (SDTS) 
with  AMDR  and  an  Aegis  Combat  System,  there  will  not 
be  a  way  to  gather  all  of  the  operationally  realistic  live  fire 
test  data  needed  for  comparison  to  accredit  the  M&S  test 
bed. 

•  Since  Aegis  employs  ESSMs  in  the  close-in,  self-defense 
battlespace,  understanding  ESSM’s  performance  is  critical 
to  understanding  the  self-defense  capabilities  of  the  DDG  5 1 
Flight  III  Destroyer. 

-  Past  DOT&E  annual  reports  have  stated  that  the  ESSM 
Block  1  operational  effectiveness  has  not  been  determined. 
The  Navy  has  not  taken  action  to  adequately  test  the 
ESSM’s  operational  effectiveness. 

-  The  IOT&E  for  ESSM  Block  2  will  be  conducted  in 
conjunction  with  the  DDG  5 1  Flight  III  Destroyer,  AMDR, 
and  Aegis  Combat  System  operational  testing. 

-  Specifically,  because  safety  limitations  preclude  ESSM 
firing  in  the  close-in  self-defense  battlespace,  there  are  very 
little  test  data  available  concerning  ESSM’s  performance, 
as  installed  on  Aegis  ships,  against  supersonic  ASCM 
surrogates. 

-  Any  data  available  regarding  ESSM’s  performance 
against  supersonic  ASCM  surrogates  are  from  a  Ship 
Self-Defense  System-based  combat  system  configuration, 
using  a  completely  different  guidance  mode  or  one  that  is 
supported  by  a  different  radar  suite. 

•  The  cost  of  building  and  operating  an  Aegis  SDTS,  estimated 
to  be  about  $350  Million,  is  small  when  compared  to  the  total 
cost  of  the  AMDR  development/procurement  and  the  eventual 


cost  of  the  22  (plus)  DDG  51  Flight  III  ships  that  are  planned 
for  acquisition  ($55+  Billion).  Even  smaller  is  the  cost  of 
the  SDTS  compared  to  the  cost  of  the  ships  that  the  DDG  5 1 
Flight  III  Destroyer  is  expected  to  protect  (approximately 
$450  Billion  in  new  ship  construction  over  the  next  30  years). 
If  DDG  5 1  Flight  III  Destroyers  are  unable  to  defend 
themselves,  these  other  ships  are  placed  at  substantial  risk. 

•  The  modification/upgrades  being  planned  for  the  DDG  5 1 
Flight  III  are  significant  enough  to  warrant  an  assessment  of 
the  impact  of  these  changes  on  ship  survivability.  The  Navy 
has  unofficially  indicated  the  DDG  51  Flight  III  LFT&E 
strategy  will  include  Component  Shock  Qualification,  a  Total 
Ship  Survivability  Trial,  and  a  Full  Ship  Shock  Trial.  Other 
LFT&E  program  particulars  are  still  under  discussion  to 
ensure  DDG  5 1  Flight  III  adequately  addresses  survivability 
requirements  against  operationally  relevant  threats  and 
recoverability  requirements. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  not 
addressed  the  following  four  previous  recommendations.  The 
Navy  should: 

1 .  Program  and  fund  an  SDTS  equipped  with  the  AMDR, 
ESSM  Block  2,  and  DDG  51  Flight  III  Aegis  Combat 
System  in  time  to  support  the  DDG  5 1  Flight  III  Destroyer 
and  ESSM  Block  2  IOT&Es. 

2.  Modify  the  AMDR,  ESSM  Block  2,  and  DDG  51  Flight 
III  Test  and  Evaluation  Master  Plans  to  include  a  phase 
of  IOT&E  using  an  SDTS  equipped  with  the  AMDR  and 
DDG  1  Flight  III  Combat  System. 

3.  Modify  the  AMDR,  ESSM  Block  2,  and  DDG  51  Flight 
III  Test  and  Evaluation  Master  Plans  to  include  a  credible 
M&S  effort  that  will  enable  a  full  assessment  of  the  AMDR, 
ESSM  Block  2,  and  DDG  51  Flight  III  Combat  System’s 
self-defense  capabilities. 

4.  Comply  with  the  DEPSECDEF  direction  to  develop 
and  fund  a  plan,  to  be  approved  by  DOT&E,  to  conduct 
at-sea  testing  of  the  self-defense  of  the  DDG  51  Flight 
III  Destroyer  with  the  AMDR,  ESSM  Block  2,  and  Aegis 
Combat  System. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Provide  program  funding  for  an  Aegis-equipped 
self-defense  test  ship  to  support  adequate  operational  testing 
of  the  AMDR,  Aegis  Combat  System,  ESSM  Block  2,  and 
DDG  5 1  Flight  III  Destroyer  programs  as  soon  as  possible 
so  as  not  to  disrupt  the  ESSM  Block  2  development 
schedule. 

2.  Provide  DOT&E  the  DDG  51  Flight  111  LFT&E  Strategy 
for  approval  prior  to  the  end  of  FY16  in  coordination  with 
the  Test  and  Evaluation  Master  Plan. 
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Department  of  the  Navy  Large  Aircraft  Infrared 
Countermeasures  (DoN  LAIRCM) 


Executive  Summary 

•  The  Navy  conducted  FOT&E  on  the  Department  of  the  Navy 
Large  Aircraft  Infrared  Countermeasure  (DoN  LAIRCM) 
Advanced  Threat  Warning  (ATW)  upgrade  installed  on  the 
CH-53E  from  December  2014  through  July  2015. 

•  DoN  LAIRCM  ATW,  as  installed  on  the  CH-53E,  provides 
new  capabilities  integrated  into  the  fielded  DoN  LAIRCM 
system,  and  is  operationally  effective  in  most  environments, 
but  not  operationally  suitable  because  of  poor  reliability  and 
logistical  supportability  issues. 

•  The  Navy  began  developmental  tests  and  operational  test 
planning  for  installation  of  DoN  LAIRCM  on  the  MV-22, 
KC-130J,  and  CH-53K  platforms. 

System 

•  The  DoN  LAIRCM  system,  a  variant  of  the  Air  Force 
LAIRCM  system,  is  a  defensive  system  for  Marine  Corps 
helicopters  designed  to  defend  against  surface-to-air  infrared 
missile  threats. 

•  DoN  LAIRCM  combines  two-color,  infrared  Missile  Warning 
Sensors  with  the  Guardian  Laser  Transmitter  Assembly 
(GLTA). 

•  The  GLTA  is  equipped  with  a  four-axis,  stabilized  gimbal 
system,  an  AN/AAR-24  Fine  Track  Sensor,  and  a  ViperTM 
laser.  The  Missile  Warning  Sensor  detects  an  oncoming 
missile  threat  and  sends  the  information  to  the  system 
processor,  which,  in  turn,  notifies  the  crew  through  the  control 
interface  unit  and  simultaneously  directs  the  GLTA  to  slew  to 
and  begin  jamming  the  threat. 

•  The  ATW  capability  upgrades  the  processor  and  missile 
warning  sensors  to  provide  improved  missile  detection  and 
adds  hostile  fire  and  laser  warning  capability  and  visual/audio 
alerts  to  the  aircrew. 

•  The  Navy  plans  to  upgrade  CF1-53E  DoN  LAIRCM  systems 
with  ATW  and  install  complete  systems  on  the  MV-22, 
KC-130J,  and  CH-53K  platforms. 

Mission 

Combatant  Commanders,  while  conducting  normal  take-off  and 

landing,  assault  landing,  tactical  descents,  re-supply,  rescue, 

forward  arming  and  refueling,  low-level  flight,  and  aerial 

refueling,  will  use  DoN  LAIRCM  and  its  new  capabilities  to: 

•  Provide  automatic  protection  of  rotary-wing  aircraft  against 
shoulder  fired,  vehicle-launched,  and  other  infrared-guided 
missiles 


•  Provide  automatic  hostile  fire  and  laser  warning  capability 
for  illuminators,  beam  riders,  laser  range  finders,  small  arms, 
rocket-propelled  grenades,  unguided  rockets,  and  anti-aircraft 
artillery 

Major  Contractor 

Northrop  Grumman,  Electronic  Systems,  Defensive  Systems 

Division  -  Rolling  Meadows,  Illinois 


ATW  -  Advanced  Threat  Warning  upgrade 
GLTA  -  Guardian  Laser  Turret  Assembly 
CIU  -  Control  Interface  Unit 


CH-53E  Helicopter 


DoN  LAIRCM 


197 


F Y1  5  NAVY  PROGRAMS 


Activity 

•  The  Navy  completed  the  ATW  engineering  and  manufacturing 
development  phase  in  October  2014  with  a  CH-53E  flight 
test  that  verified  software  fixes  to  failures  that  were  identified 
during  earlier  developmental  tests. 

•  The  Navy  conducted  FOT&E  on  the  DoN  LAIRCM  ATW 
upgrade  installed  on  the  CH-53E  from  December  2014 
through  July  2015  to  support  a  fielding  decision  for  ATW. 

•  During  FOT&E,  the  Commander,  Operational  Test  and 
Evaluation  Force  collected  effectiveness  and  suitability  data 
from  the  following  tests: 

-  Shipboard  compatibility  tests  near  and  onboard  the  USS 
Bataan 

-  Flight  tests  at  Eglin  AFB,  Florida,  and  Marine  Corps  Air 
Station  (MCAS)  New  River,  North  Carolina 

-  False  alert  tests  during  a  deployment  to/from  Yuma, 
Arizona 

-  A  maintenance  demonstration  at  MCAS  New  River 

-  A  cybersecurity  test  at  MCAS  New  River 

•  The  Navy  began  developmental  tests  and  operational  test 
planning  for  installation  of  DoN  LAIRCM  on  the  MV-22, 
KC-130J,  and  CH-53K  platforms. 

•  In  early  2016,  DOT&E  is  expected  to  submit  an  FOT&E 
report  on  DoN  LAIRCM  detailing  the  results  of  testing  the 
system  on  the  CH-53E  platform. 

Assessment 

•  DoN  LAIRCM  ATW,  as  installed  on  the  CH-53E,  provides 
new  capabilities  integrated  into  the  fielded  DoN  LAIRCM 
system,  and  is  operationally  effective  in  most  environments, 
but  it  is  not  operationally  suitable  because  of  reliability  and 
logistical  supportability  issues. 

-  The  operational  effectiveness  for  hostile  fire  indication 
and  laser  warning  exceeded  the  classified  threshold 
requirements,  while  the  missile  warning  and  infrared 
countermeasures  capabilities  were  not  degraded  by  the 
ATW  upgrade. 

-  The  ATW  system’s  Mean  Time  Between  Operational 
Mission  Failure  was  26  hours  with  an  80  percent 
confidence,  which  is  well  below  the  threshold  requirement 
of  130  hours. 

-  Logistic  supportability  shortfalls  are  as  follow: 

■  The  location  of  the  control  interface  unit  is  poorly 

situated  and  not  useful  during  combat  maneuvering  since 
its  current  location  requires  the  pilot  to  look  inside  the 
cockpit  during  critical  flight  regimes. 


■  Technical  documentation  and  training  regarding  the 
operational  employment  aspects  of  in-flight  power 
cycles  need  updating  to  avoid  inadvertent  loss  of  system 
availability  during  flight. 

■  Obsolescence  of  the  smart  card  utility  software  and  the 
smart  cards  themselves  will  likely  cause  current/ future 
maintenance  and  reprogramming  challenges  for  the  DoN 
LAIRCM  system. 

•  ATW  effectiveness  in  denied-GPS  environments  (or  with 
GPS  jamming)  is  unknown.  GPS  loss  was  not  envisioned  to 
impact  the  system;  however,  during  a  flight  near  the  end  of  the 
test  program,  an  unplanned  loss  of  GPS  caused  some  system 
anomalies  that  post-test  analysis  revealed  traceable  to  aircraft 
GPS  loss. 

•  While  limited  in  scope,  the  cybersecurity  assessment  was 
sufficient  for  the  federated  installation  on  the  CFI-53E.  A 
Cooperative  Vulnerability  and  Penetration  Assessment 
identified  a  vulnerability  in  the  laptop  computer  which  is  used 
to  program  ATW  sensors. 

•  All  testing  was  conducted  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and 
FOT&E  test  plan  to  support  a  fielding  decision  for  the  ATW 
upgrade  on  the  CFI-53E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  satisfactorily 
addressed  all  previous  recommendations  on  the  legacy 

DoN  LAIRCM  program.  This  is  the  first  annual  report  on  the 
ATW  upgrade. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Continue  to  improve  reliability  of  the  ATW  sensors,  and 
monitor  and  report  reliability  growth  to  DOT&E. 

2.  Resolve  the  logistic  supportability  obsolescence  problems 
with  the  smart  cards  used  to  operate,  maintain,  and 
reprogram  the  DoN  LAIRCM  system. 

3.  Resolve  the  logistic  supportability  and  human  factors 
problem  with  the  location  of  the  control  indicator  unit. 

4.  Resolve  the  logistic  supportability  shortfall  in  the  technical 
documentation  and  training  regarding  operational 
employment  aspects  of  in-flight  power  cycles. 

5.  Collect  effectiveness  data  in  a  denied-GPS  or  GPS-jammed 
environment  during  FOT&E  on  either  the  MV-22  or 
KC-130J  installations  on  DoN  LAIRCM. 
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E-2D  Advanced  Hawkeye 


Executive  Summary 

•  The  Commander,  Operational  Test  and  Evaluation  Force 
(COTF)  conducted  the  first  E-2D  Advanced  Hawkeye  FOT&E 
period  in  2QFY14  to  evaluate  the  E-2D  Initial  Operational 
Capability  hardware  and  software  configuration:  Delta 
System/Software  Configuration  (DSSC)  Build  1.  COTF 
completed  testing  in  3QFY15. 

•  FOT&E  showed  the  E-2D  had  no  significant  performance 
differences  compared  to  IOT&E  and  was  adequate  to  assess 
E-2D  suitability  and  effectiveness  for  legacy  E-2C  missions. 
Unlike  the  IOT&E,  FOT&E  included  adequate  E-2D  carrier 
testing.  An  evaluation  on  E-2D’s  capability  to  perform  the 
Theater  Air  and  Mission  Defense  (TAMD)  mission  cannot  be 
made  until  future  FOT&E  periods.  DOT&E  will  submit  an 
FOT&E  report  on  the  E-2D  in  2QFY16. 

•  Change  1  to  the  E-2D  Test  and  Evaluation  Master  Plan 
(TEMP)  revision  D  supports  the  second  FOT&E  period, 
which  includes  requirements  or  resources  for  integration  and 
operational  testing  of  Naval  Integrated  Fire  Control-Counter 
Air  (NIFC-CA)  From  the  Air  (FTA).  Change  1  will  address 
NIFC-CA  FTA  areas  relevant  to  E-2D  only,  and  to  support 
DSSC  Build  2  in  3QFY16  for  FOT&E. 

System 

•  The  E-2D  is  a  carrier-based  Airborne  Early  Warning  and 
Command  and  Control  aircraft. 

•  Significant  changes  to  this  variant  of  the  E-2  include  upgraded 
engines  to  provide  increased  electrical  power  and  cooling 
relative  to  current  E-2C  aircraft;  a  strengthened  fuselage  to 
support  increased  aircraft  weight;  replacement  of  the  radar 
system,  the  communications  suite,  and  the  mission  computer; 
and  the  incorporation  of  an  all-glass  cockpit,  which  permits 
the  co-pilot  to  act  as  a  tactical  fourth  operator  in  support  of  the 
system  operators  in  the  rear  of  the  aircraft. 

•  The  radar  upgrade  replaces  the  E-2C  mechanically  scanned 
radar  with  a  phased-array  radar  that  has  combined  mechanical 
and  Electronically  Scanned  Array  capabilities. 


•  The  upgraded  radar  is  intended  to  provide  significant 
improvement  in  littoral  and  overland  detection  performance 
and  TAMD  capabilities. 

•  The  E-2D  Advanced  Hawkeye  Integrated  Training  System 
includes  all  simulators,  interactive  computer  media,  and 
documentation  to  conduct  maintenance,  as  well  as  aircrew 
shore -based  initial  and  follow-on  training. 

Mission 

The  Combatant  Commander,  whether  operating  from  the  aircraft 

carrier  or  from  land,  will  use  the  E-2D  Advanced  Hawkeye  to 

accomplish  the  following  missions: 

•  Theater  air  and  missile  sensing  and  early  warning 

•  Battlefield  management,  command,  and  control 

•  Acquisition,  tracking,  and  targeting  of  surface  warfare  contacts 

•  Surveillance  of  littoral  area  objectives  and  targets 

•  Tracking  of  strike  warfare  assets 

Major  Contractor 

Northrop  Grumman  Aerospace  Systems  -  Melbourne,  Florida 


Activity 

•  COTF  conducted  E-2D  carrier  testing,  a  major  shortfall  of 
the  IOT&E,  in  September  and  October  2014,  onboard  the 
USS  Theodore  Roosevelt. 

•  In  2QFY14,  COTF  conducted  the  E-2D’s  first  FOT&E  period 
to  assess  the  operational  effectiveness  and  suitability  of 
hardware  and  software  changes  incorporated  in  DSSC  Build  1 
and  to  support  the  TAMD  mission.  The  Navy  conducted 
testing  at  Naval  Air  Station  (NAS)  Patuxent  River,  Maryland; 
Holloman  AFB,  New  Mexico;  White  Sands  Missile  Range, 


New  Mexico;  NAS  Point  Mugu,  California;  NAS  Norfolk, 
Virginia;  USS  Theodore  Roosevelt  (CVN  71);  and  Naval 
Air  Weapons  Station  China  Lake,  California,  in  accordance 
with  the  DOT&E-approved  TEMP  and  test  plan.  FOT&E 
completed  in  3QFY15. 

•  Per  the  FOT&E  test  plan,  the  Navy  only  conducted  a 
qualitative  assessment  on  E-2D  training  devices  (such  as 
simulators)  and  student  training  time.  The  second  FOT&E 
period  will  have  a  full  training  assessment. 
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•  In  2QFY16,  DOT&E  will  submit  an  FOT&E  report  on  the 
E-2D. 

•  Change  1  to  the  E-2D  TEMP  revision  D  supports  the  second 
FOT&E  period,  which  includes  requirements  for  resources  and 
integration  of  operational  testing  of  N1FC-CA  FTA.  Change 

1  will  address  NIFC-CA  FTA  areas  relevant  to  E-2D  only,  and 
to  support  DSSC  Build  2  in  3QFY16  for  FOT&E.  Change 
1  will  also  include  cybersecurity  testing  in  accordance  with 
DOT&E ’s  guidelines  detailed  in  Procedures  for  Operational 
Test  and  Evaluation  of  Cybersecurity  Acquisition  Programs, 
dated  August  1,  2014. 

•  The  Navy  continues  to  correct  deficiencies  with  E-2D 
Cooperative  Engagement  Capability  performance  with  a  plan 
to  have  deficiencies  remedied  in  1QFY19  with  fielding  of 
DSSC  Build  3. 

Assessment 

•  E-2D  carrier  suitability  performance  is  similar  to  the  aircraft’s 
performance  when  it  is  disembarked. 

•  FOT&E  showed  the  E-2D  has  no  significant  performance 
differences  compared  to  IOT&E,  but  has  similar  shortfalls 
on  some  radar  reliability,  availability,  and  weapon  system 
metrics.  FOT&E  was  adequate  to  assess  E  2D  suitability  and 
effectiveness  for  legacy  E-2C  missions.  An  evaluation  on 
E-2D’s  capability  to  perform  the  TAMD  mission  cannot  be 
made  until  future  FOT&E  periods. 

•  DOT&E’s  classified  report  will  explain  all  findings  in  more 
detail. 


•  Change  1  to  the  E-2D  TEMP  revision  D  will  address 
requirements  and  resources  for  integrated  or  operational 
testing  of  NIFC-CA  FTA  and  will  include  more  evolved 
cybersecurity  testing.  DOT&E  is  providing  cybersecurity 
guidance  to  Change  1  and  all  subsequent  TEMPs  for  future 
FOT&E  periods. 

•  A  full  assessment  of  E-2D  operational  capabilities  will  require 
future  FOT&Es  and  systematic  updates. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  continues  to 
improve  radar  and  mission  system  performance  and  to  improve 
radar  and  overall  weapon  system  reliability  and  availability  as 
previously  recommended. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Provide  complete  training  on  all  components  of  the  E-2D 
system  and  missions. 

2.  Complete  the  Change  1  to  the  E-2D  TEMP  revision  D  and 
test  plan. 

3.  Incorporate  DOT&E  guidance  in  its  cybersecurity  testing 
for  all  subsequent  FOT&E  periods. 

4.  Improve  radar  and  overall  weapon  system  reliability  and 
availability. 

5.  Continue  to  correct  E-2D  Cooperative  Engagement 
Capability  performance  deficiencies. 
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F/A-18E/F  Super  Hornet  and  EA-18G  Growler 


Executive  Summary 

•  The  Navy  conducted  Software  Qualification  Testing  of 
System  Configuration  Set  (SCS)  H10E  for  the  F/A-18E/F 
Super  Hornet  and  EA-18G  Growler  aircraft  with  fleet  release 
planned  for  early  2016.  Major  upgrades  tested  during 

this  period  for  the  Super  Hornet  include  improvements 
to  multi-sensor  integration,  aircrew  displays,  short-range 
tracking,  and  combat  identification.  Growler  improvements 
tested  include  the  Joint  Tactical  Terminal  Receiver,  enhanced 
combat  identification  capabilities,  and  expanded  jamming 
assignments. 

•  The  F/A-18E/F  Super  Hornet  weapon  system  continues 
to  demonstrate  operational  effectiveness  for  most  threat 
environments;  however,  the  platform  is  not  operationally 
effective  in  specific  threat  environments,  which  are  detailed  in 
previous  DOT&E  classified  reports. 

•  Although  the  reliability  of  the  APG-79  radar  has  improved 
over  multiple  phases  of  operational  test,  Active  Electronically 
Scanned  Array  (AESA)  radar  software  stability  problems 
resulted  in  the  continued  failure  of  the  radar  to  meet  reliability 
and  built-in  test  (BIT)  performance  requirements.  DOT&E 
expects  improved  AESA  reliability  with  SCS  H10. 

•  DOT&E  previously  assessed  the  EA-18G  Growler  weapons 
system  equipped  with  SCS  H8E  as  operationally  suitable  and 
operationally  effective  with  the  same  radar  limitations  as  the 
F/A-18E/F. 

•  The  Navy  completed  SCS  25X  testing  in  4QFY15  for 
non-Higher  Order  Language  (HOL)  software  on  F/A-18s 
and  release  to  the  fleet  in  2QFY16.  Early  lot  F/A-18E/Fs 
will  remain  in  the  Non-HOL  configuration  (SCS  25X)  until 
mission  computers  are  upgraded  to  operate  HOL. 

System 

F/A-18E/F  Super  Hornet 

•  The  Super  Hornet  is  the  Navy’s  premier  strike-fighter 
aircraft  and  is  a  more  capable  follow-on  replacement  to  the 
F/A-18A/B/C/D  and  the  F-14. 

•  F/A-18E/F  Lot  25+  aircraft  provide  functionality  essential 
for  integrating  all  Super  Hornet  Block  2  hardware  upgrades, 
which  include: 

-  Single  pass  multiple  targeting  for  GPS-guided  weapons 

-  Use  of  off-board  target  designation 

-  Improved  datalink  for  target  coordination  precision 

-  Implementation  of  air-to-ground  target  aim  points 

•  Additional  systems  include: 

-  APG-73  or  APG-79  radar 

-  Advanced  Targeting  Forward-Looking  Infrared  System 

-  AIM-9  infrared-guided  missiles  and  AIM- 120  and  AIM-7 
radar-guided  missiles 

-  Multi-functional  Information  Distribution  System  for 
Link  16  tactical  datalink  connectivity 


-  Joint  Helmet-Mounted  Cueing  System 

-  Integrated  Defensive  Electronic  Countermeasures 

EA-18G  Growler 

•  The  Growler  is  the  Navy’s  land-  and  carrier-based,  radar  and 
communication  jamming  aircraft. 

•  The  two-seat  EA-18G  replaces  the  four-seat  EA-6B.  The  new 
ALQ-218  receiver,  improved  connectivity,  and  linked  displays 
are  the  primary  design  features  implemented  to  reduce  the 
operator  workload  in  support  of  the  EA-18G’s  two-person 
crew. 

•  The  Airborne  Electronic  Attack  system  includes: 

-  Modified  EA-6B  Improved  Capability  III  ALQ-218 
receiver  system 

-  Advanced  crew  station 

-  Legacy  ALQ-99  jamming  pods 

-  Communication  Countermeasures  Set  System 

-  Expanded  digital  Link  16  communications  network 

-  Electronic  Attack  Unit 

-  Interference  Cancellation  System  that  supports 
communications  while  jamming 

-  Satellite  receive  capability  via  the  Multi-mission  Advanced 
Tactical  Terminal 

•  Additional  systems  include: 

-  APG-79  AESA  radar 

-  Joint  Helmet-Mounted  Cueing  System 

-  High-speed  Anti-Radiation  Missile 

-  AIM-120  radar-guided  missiles 
System  Configuration  Set  (SCS)  Software 

•  Growler  and  Super  Hornet  aircraft  employ  SCS  operational 
software  to  enable  major  combat  capabilities.  All  EA-18Gs 
and  Block  2  F/A-18s  (production  Lot  23  and  beyond)  use 
high-order  language  or  “H-series”  software,  while  F/A-18E/F 
prior  to  Lot  23  and  all  legacy  F/A-18  A/B/C/D  aircraft  use 
“X-series”  software. 
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Mission 

•  Combatant  Commanders  use  the  F/A-18E/F  to: 

-  Conduct  offensive  and  defensive  air  combat  missions 

-  Attack  ground  targets  with  most  of  the  U.S.  inventory  of 
precision  and  non-precision  weapon  stores 

-  Provide  in-flight  refueling  for  other  tactical  naval  aircraft 

-  Provide  the  fleet  with  an  organic  tactical  reconnaissance 
capability 

•  Combatant  Commanders  use  the  EA-18G  to: 

-  Support  friendly  air,  ground,  and  sea  operations  by 
countering  enemy  radar  and  communications 

-  Jam  integrated  air  defense  systems 

-  Support  non-integrated  air  defense  missions  and  emerging 
non-lethal  target  sets 


-  Enhance  crew  situational  awareness  and  mission 
management 

-  Enhance  connectivity  to  national,  theater,  and  tactical  strike 

assets 

-  Provide  enhanced  lethal  suppression  through  accurate 
High-speed  Anti-Radiation  Missile  targeting 

-  Provide  the  EA-18G  crew  air-to-air  self-protection  with  the 
AIM- 120 

Major  Contractor 

The  Boeing  Company,  Integrated  Defense  Systems  -  St.  Louis, 
Missouri 


Activity 

•  The  Navy  conducted  F/A-18E/F  and  EA-18G  H10  Software 
Qualification  Testing  from  March  through  September  2015  in 
accordance  with  a  DOT&E-approved  test  plan  and  in  support 
of  fleet  introduction  in  2QFY16.  Test  aircraft  equipped  with 
SCS  H10E  accumulated  1,820  flight  hours. 

•  Major  upgrades  tested  during  this  period  for  the  Super  Hornet 
include  improvements  to  multi-sensor  integration,  aircrew 
displays,  short-range  tracking,  and  combat  identification. 

•  Growler  improvements  tested  include  the  Joint  Tactical 
Terminal  Receiver,  enhanced  combat  identification 
capabilities,  and  expanded  jamming  assignments. 

•  The  Navy  has  continued  to  defer  development  of  the  AESA’s 
full  electronic  warfare  capability  to  later  software  builds  and 
plans  to  test  this  capability  in  SCS  H12  and  H14. 

•  Early  lot  F/A-18E/Fs  will  remain  in  the  Non-HOL 
configuration  (SCS  25X)  until  mission  computers  are 
upgraded  to  operate  HOL. 

•  The  Navy  completed  SCS  25X  testing  in  4QFY15  for  earlier 
lot  F/A-18s  and  is  expected  to  begin  releasing  the  non-HOL 
software  to  the  fleet  in  1QFY16. 

Assessment 

•  DOT&E  does  not  expect  test  results  from  SCS  H10E  FOT&E 
to  alter  the  previous  assessment  made  in  F/A-18E/F  and 
EA-18G  reports  regarding  key  deficiencies  in  operational 
performance.  While  both  systems  remain  operationally 
effective  in  some  threat  environments,  DOT&E  has  noted  in 
classified  reports  that  both  systems  remain  not  effective  in  the 
more  stressful  current  air  warfare  environments.  Though  the 
Navy  has  begun  to  address  long-standing  deficiencies  in  air 
warfare  by  making  incremental  improvements  in  capability 
during  recent  FOT&E  periods,  DOT&E  is  unlikely  to  change 
this  finding  until  deficiencies  are  resolved. 


•  Early  results  from  SCS  H10  testing  suggest  improvements 
in  AESA  reliability.  The  Navy  has  made  incremental 
improvements  in  AESA  radar  reliability  since  the 

2006  IOT&E.  Radar  software  instability  has  resulted  in  the 
program’s  failure  to  meet  reliability  and  BIT  performance 
requirements  in  every  test  period.  The  AESA  provides 
improved  performance  compared  to  the  legacy  APG-73  radar 
used  on  older  F/A-18E/F  aircraft. 

•  DOT&E  determined  that  the  E A- 1 8G  Growler  was 
operationally  effective  and  suitable,  though  subject  to  the 
same  threat  limitations  as  the  Super  Hornet,  noting  a  need  for 
improved  jammer  timeliness.  SCS  H10E  FOT&E  results  are 
not  likely  to  change  this  assessment. 

•  SCS  H8E  testing  did  not  include  an  end-to-end  multi- 
AIM-120  missile  shot.  This  Navy  operational  capability  has 
not  been  demonstrated  previously  in  a  successful  test.  The 
Navy  has  agreed  to  include  a  multi-missile  shot  during  SCS 
H12  testing. 

Recommendations 

•  Status  of  Previous  Recommendations.  Per  previous 
recommendations,  the  Navy  should  continue  to  improve 
the  reliability  and  BIT  functionality  of  the  AESA  radar  and 
develop  and  characterize  the  full  electronic  warfare  capability 
of  the  APG-79  radar.  DOT&E’s  recommendation  to  conduct 
an  operationally  representative  end-to-end  missile  test  to 
demonstrate  APG-79  radar  and  system  software  support  for  a 
multiple  AIM- 120  missile  engagement  also  remains. 

•  FY15  Recommendation. 

1.  The  Navy  should  focus  on  improvements  that  will  allow  the 
Super  Hornet  and  Growler  to  be  operationally  effective  in 
all  threat  environments. 
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Global  Combat  Support  System  -  Marine  Corps 

(GCSS-MC) 


Executive  Summary 

•  The  Marine  Corps  Operational 
Test  and  Evaluation  Activity 
(MCOTEA)  conducted  the 
FOT&E  of  the  Global  Combat 
Support  System  -  Marine  Corps 
(GCSS-MC)  Release  1.1.1  from 
August  through  November  2014, 
in  support  of  a  Full  Deployment 
Decision  in  early  2015.  The 
test  was  conducted  in  Camp 
Lejeune,  North  Carolina  and 
Okinawa,  Japan,  with  actual 
users  in  a  live  environment. 

•  Of  the  three  enhancements 
included  in  Release  1.1.1,  two 
were  operationally  effective  and 
one  was  not.  One  enhancement 


CC/JTF 


Adjacent/Subordinate 
and  Attached  Allied,  Joint 
and  other  Service  Units 


Defense  Logistic 
Agency 

Supporting 

Establishment 

Vendors  & 
Commercial  Industry 

Host  Nation  Support 


CC/JTF:  Combatant  Commander/Joint  Task  Force 
GCSS-MC/LCM:  Global  Combat  Support 

System  -  Marine  Corps/Logistics  Chain  Managemer 
HQMC  l&L:  Headquarters  Marine  Corps  Installation 
and  Logistics 

MAGTF:  Marine  Air  Ground  Task  Force 
MEFs:  Marine  Expeditionary  Force 
NLI:  Naval  Logistics  Integration 


GCSS-MC/LCM  Provides: 
•Visibility  of  Logistics  Execution 
•Enterprise  Logistics  Planning 
•Logistics  Command  &  Control 
•Logistics  Chain  Management 


was  operationally  suitable,  but 
two  were  not. 

•  The  GCSS-MC  Program  Manager  has  taken  action  to  address 
the  issues  found  during  FOT&E. 

•  In  March  2015,  USD(AT&L)  granted  a  Full  Deployment 
Decision. 


•  GCSS-MC  Release  1.1.1  provides  three  capability 
enhancements  to  the  currently  fielded  logistics  system: 
Enterprise  Automated  Task  Organization  (EATO),  Mobile 
Field  Service  (MFS),  and  Tactical- Wide  Area  Network 
(T-WAN)  optimization 


System 

•  GCSS-MC  is  the  Marine  Corps  component  of  the  joint 
GCSS  Family  of  Systems.  It  is  intended  to  provide  a 
seamless  end-to-end  logistics  chain  requiring  less  reliance 
on  forward-positioned  materiel,  and  capitalizing  on  the 
availability  of  near  real-time  logistics  information  critical 
to  supported  and  supporting  units.  GCSS-MC  includes  all 
transactional  Combat  Service  Support  systems  related  to 
supply  chain  management  and  enterprise  asset  management 
functionality,  enabled  with  service  management  functions. 

•  GCSS-MC  is  a  commercial  off-the-shelf  Enterprise  Resource 
Planning  system  that  uses  the  Oracle  E-Business  Suite  to 
provide  logistics  chain  management. 

•  GCSS-MC  provides  the  Marine  Corps  with  authoritative 
supply,  maintenance,  and  transportation  data  in  DOD  data 
dictionary-compliant  format. 


Mission 

•  Combatant  Commanders/Joint  Task  Force  Commanders 
will  use  GCSS-MC  to  provide  an  end-to-end  logistics  chain 
that  decreases  reliance  on  forward-positioned  materiel  and 
capitalizes  on  the  availability  of  near  real-time  logistics 
information  critical  to  supported  and  supporting  units. 

•  Marines  operating  in  garrison  and  deployed  Marine 
Air-Ground  Task  Force  units  will  use  GCSS-MC  to  conduct 
logistics  missions  that  support  Marine  Corps  operations. 

Major  Contractor 

Oracle  Corporation  -  Reston,  Virginia 


Activity 

•  MCOTEA  developed  and  DOT&E  approved  an  operational  Camp  Lejeune,  North  Carolina,  in  accordance  with  a 

test  plan  for  GCSS-MC  FOT&E  in  August  2014.  DOT&E-approved  test  plan.  Testing  supported  an  early  2015 

•  MCOTEA  conducted  the  FOT&E  from  August  Full  Deployment  Decision  and  was  tied  to  the  Philippines 

through  November  2014  in  Okinawa,  Japan,  and 
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Bilateral  Exercise  and  the  Bold  Alligator  Exercise.  Marines 
from  II  Marine  Expeditionary  Force  (MEF)  and  III  MEF 
participated  in  the  test. 

•  GCSS-MC  Release  1.1.1  provides  three  capability 
enhancements  to  the  currently  fielded  logistics  system: 
Enterprise  Automated  Task  Organization  (EATO),  Mobile 
Field  Service  (MFS),  and  Tactical- Wide  Area  Network 
(T-WAN)  optimization.  MCOTEA  conducted  separate  events 
during  the  FOT&E  to  evaluate  each  of  the  enhancement 
solutions. 

•  DOT&E  submitted  an  FOT&E  report  in  February  2015 
detailing  the  results  of  testing. 

•  In  April  2015,  the  GCSS-MC  Program  Manager  conducted 
MFS  afloat  demonstration  testing  aboard  the  USS  Essex 
and  USS  Rushmore  to  provide  additional  data  that  were  not 
collected  during  FOT&E. 

•  In  March  2015,  USD(AT&L)  granted  a  Full  Deployment 
Decision. 

Assessment 

•  Based  on  FOT&E  results  DOT&E  assessed  the  system  as 
follows: 

-  EATO  is  operationally  effective.  EATO  performed  as 
expected  when  properly  operated,  but  lack  of  training 
and  documentation  led  to  user  errors.  EATO  is  not 
operationally  suitable  and  required  improved  training  and 
documentation. 

-  T-WAN  optimization  is  operationally  effective  and 
operationally  suitable.  T-WAN  optimization  reduced  the 
data  size  for  different  transactions.  Users  were  able  to 
complete  all  tasks  and  System  Usability  Scale  (survey 
scores  were  favorable). 

-  MFS  is  not  operationally  effective  and  not  operationally 
suitable.  Implementation  problems  and  lack  of  adequate 


training  and  documentation  prevented  some  users  from 
using  the  system  as  designed.  At  the  time  of  test,  Public 
Key  Infrastructure-enabled  MFS  was  unavailable  and 
the  users  required  strong,  complex  passwords.  This  led 
to  frustration  by  the  users  because  the  passwords  were 
difficult  to  enter  and  the  system  did  not  provide  sufficient 
feedback  when  generating  and  inputting  passwords. 
Additionally,  this  led  to  accidental  account  lockout 
and  inability  to  complete  tasks.  The  help  desk  had  no 
knowledge  of  the  system  and  was  unable  to  provide  any 
assistance.  Users  did  not  rate  the  system  favorably  using 
the  System  Usability  Scale,  and  their  favorability  ratings 
decreased  as  testing  progressed. 

•  The  GCSS-MC  Program  Manager  has  taken  action  to  address 
the  issues  found  during  FOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Marine  Corps 
addressed  all  previous  recommendations. 

•  FY 15  Recommendations.  The  Marine  Corps  should: 

1 .  Implement,  document,  and  monitor  the  process  to  train  and 
educate  the  operating  forces  in  a  manner  that  will  allow  for 
users  to  properly  operate  the  GCSS-MC  Rl.1.1.  Confirm 
that  such  training  and  documentation  are  in  place  before  full 
fielding  of  GCSS-MC  Rl.1.1. 

2.  Address  issues  identified  in  the  FOT&E  Test  Incident 
Reports  and  field  a  Public  Key  Infrastructure-enabled  MFS 
to  eliminate  the  need  for  complex  passwords. 

3.  Conduct  a  cybersecurity  Adversarial  Assessment  of 
GCSS-MC.  Ensure  cybersecurity  testing  includes  the  2015 
Oracle  E-Business  System  R12  upgrade. 
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H-1  Upgrades  -  U.S.  Marine  Corps  Upgrade  to  AH-1Z 
Attack  Helicopter  and  UH-1Y  Utility  Helicopter 


Executive  Summary 

•  The  Navy  conducted  OT-IIIC  FOT&E,  from  March  through 
May  2015,  focused  on  the  evaluation  of  the  aircraft  System 
Configuration  Set  (SCS)  7.1  software,  which  is  designed 

to  enhance  capabilities  and  correct  previously  identified 
deficiencies. 

•  The  H-1  Upgrade  aircraft  with  SCS  7. 1  remains  operationally 
effective  and  survivable.  Pilots  provided  positive  feedback 
on  all  enhancements  except  for  the  accuracy  of  geographic 
coordinates  provided  by  the  Brite  Star  Block  II  sensor  on  the 
UH-1Y. 

•  The  new  SCS  7. 1  virtual  targeting  reticle  reduces  pilot 
workload  for  employment  of  the  Advanced  Precision  Kill 
Weapon  System  (APKWS).  Expansion  of  the  reticle’s 
indicating  envelope  is  needed. 

•  APKWS  performance  was  consistent  with  requirements. 
Improvements  are  needed  in  documentation  and  markings 
as  well  as  enhancements  in  training  to  avoid  inadvertently 
activating  a  special  mode  during  preflight  preparations. 

•  H-1  Upgrade  aircraft  are  not  suitable  due  to  unsatisfactory 
measures  of  availability  based  on  Full  Mission  Capable 
(FMC)  and  Mission  Capable  (MC)  rates.  OT-IIIC  aircraft 
demonstrated  a  63  percent  MC  rate  because  of  long  downtimes 
awaiting  repair  parts. 

System 

•  This  program  upgrades  two  Marine  Corps  H-1  aircraft: 

-  The  AH-1 W  Attack  Helicopter  is  upgraded  to  the  AH-1Z 

-  The  UH-1N  Utility  Helicopter  is  upgraded  to  the  UH-1Y 

•  The  aircraft  have  identical  twin  engines,  drive  trains,  four 
bladed  rotors,  tail  sections,  digital  cockpits,  and  helmet 
mounted  sight  displays.  By  parts  count,  the  aircraft  are 
84  percent  common. 

•  The  UH-1Y  has  twice  the  payload  and  range  of  legacy 
UH-1N  aircraft  and  can  deliver  eight  combat-ready  Marines 
118  nautical  miles  and  return  without  refueling. 

•  The  AH-  1Z  has  improved  payload  and  a  high-fidelity  targeting 
sensor  for  delivery  of  air  to  ground  and  air-to-air  missiles, 
rockets,  and  guns. 

•  SCS  7.1  provides  a  virtual  targeting  reticle  for  employment 
of  the  APKWS  from  both  aircraft.  APKWS  provides  a 


UH-1 Y  (rear)  and  AH-1Z  (far) 


laser-guidance  section  for  more  precise  employment  of  the 
2.75-inch  rocket. 

•  As  of  October  2015,  Bell  Helicopter  has  delivered  123  of  the 
planned  160  UH-1Y  aircraft  and  42  of  the  planned  189  AH-1Z 
aircraft. 

Mission 

Marine  light/attack  helicopter  squadron  detachments  are  typically 

deployed  with  a  mix  of  UH-1  Y  and  AH-1Z  helicopters.  During 

these  missions: 

•  Detachments  equipped  with  the  AH-1Z  use  the  attack 
helicopter  to  conduct  rotary-wing  close  air  support,  anti-armor, 
armed  escort,  armed  and  visual  reconnaissance,  and  fire 
support  coordination  missions. 

•  Detachments  equipped  with  the  UH-1Y  use  the  utility 
helicopter  to  conduct  command,  control,  assault  support, 
escort,  air  reconnaissance,  and  aeromedical  evacuation 
missions. 

Major  Contractor 

Bell  Helicopter  -  Amarillo,  Texas 


Activity 

•  The  Navy  conducted  OT-IIIC  FOT&E  of  the  AH- 1 Z 
and  UH-1Y  aircraft  from  March  through  May  2015  at 
Marine  Corps  Air  Station,  Yuma,  Arizona;  Naval  Air 


Station,  Fallon,  Nevada;  Marine  Corps  Base  Twenty-nine 
Palms,  Camp  Pendleton;  and  Naval  Aviation  Weapons  Station 
China  Lake,  California. 
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•  The  Navy  conducted  testing  in  accordance  with  the 
DOT&E-approved  test  plan  dated  March  15,  2015. 

•  OT-IIIC  focused  on  the  evaluation  of  the  newly  installed 
SCS  7.1  software,  which  improves  the  management  and 
presentation  of  mission  and  geographic  data  to  pilots,  and 
corrects  previously  identified  deficiencies. 

•  Four  aircraft  (two  AH-lZs,  and  two  UH-lYs)  completed  a 
total  of  205  flight  hours  in  integrated  and  operational  testing. 
Crews  completed  18  operational  missions  during  operational 
testing  in  realistic  desert  environments  against  simulated  threat 
targets. 

Assessment 

•  H-l  Upgrade  units  with  SCS  7. 1 -equipped  aircraft  remain 
operationally  effective.  The  test  unit  successfully  completed 
18  of  18  planned  tactical  missions.  Pilots  provided  positive 
feedback  on  all  SCS  7.1  enhancements  over  SCS  6.0  software. 

•  Geographic  coordinates  provided  by  the  Brite  Star  Block  II 
sensor  on  the  UH-1Y  were  useful  to  assist  with  basic  aircraft 
navigation  and  pilot  situational  awareness.  While  not  a 
requirement,  Brite  Star  Sensor  Block  II  coordinates  were  not 
accurate  enough  for  weapons  employment. 

•  The  virtual  targeting  reticle  reduces  pilot  workload  for 
APKWS  employment  while  the  aircraft  is  within  the 
parameters  for  APKWS  launch.  Expansion  of  the  reticle’s 
indicating  range  is  needed  beyond  the  current  APKWS 
launch  envelope  because  whenever  the  aircraft  is  outside  that 
envelope  in  any  parameter,  the  reticle  parks  on  the  aircraft 
datum  line  and  stays  there.  Pilots  complained  the  parked 
reticle  provides  no  information  on  how  to  get  back  within 
the  launch  window,  which  can  hinder  the  ability  to  fire  the 
APKWS  accurately  in  that  situation. 

•  APKWS  performance  on  both  UH-1Y  and  AH-1Z  was 
consistent  with  requirements.  Improvements  are  needed 
in  maintenance  crew  training  and  documentation  on  the 
mechanisms  for  making  preflight  mode  selections  on  the 
rockets  (which  cannot  be  changed  after  take-off).  Six  APKWS 
shots  missed  when  crewmen  unintendedly  enabled  the 
counter-countermeasure  mode. 

•  OT-IIIC  aircraft  met  reliability  and  maintainability 
requirements. 

-  UH-1Y  demonstrated  reliability  of  5.0  mean  flight 
hours  between  failures  against  a  requirement  of  being 
greater  than  or  equal  to  0.9  hours  and  demonstrated 
3.5  unscheduled  maintenance  man  hours  per  flight  hour 
against  a  requirement  of  being  less  than  or  equal  to 

3.9  hours. 

-  AH-  1Z  demonstrated  reliability  of  2.5  mean  flight  hours 
between  failures  against  a  requirement  of  being  greater 


or  equal  to  0.8  hours  and  demonstrated  maintainability 
of  4.3  unscheduled  maintenance  man  hours  per  flight 
hour  against  a  requirement  of  being  less  than  or  equal  to 
4.3  hours. 

•  OT-IIIC  and  fleet  aircraft  showed  unsatisfactory  results 
for  measures  of  availability  based  on  FMC  and  MC  rates, 
resulting  from  unsatisfactory  logistics  supportability. 

H-l  Upgrade  aircraft  did  not  meet  the  availability  requirement 
of  being  85  percent  MC.  The  OT-IIIC  unit  demonstrated 
63  percent  MC  because  of  long  downtimes  awaiting  repair 
parts.  The  longest  delay  time  for  high-priority  parts  delivery 
during  OT  IIIC  was  61.82  days.  One  sixth  of  all  high-priority 
components  ordered  had  a  logistic  delay  time  in  excess 
of  10  days.  Nineteen  critical  components  that  failed  were 
cannibalized  from  other  aircraft  and/or  units  in  order  to 
facilitate  operational  test.  All  of  these  components  were 
related  to  the  fielded  aircraft  and  were  not  attributable  to  SCS 
7.1  enhancements. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
progress  addressing  the  previous  FY13  recommendation  to 
improve  self-sealing  capabilities  of  fuel  bladders  but  still 
needs  to: 

1 .  Initiate  the  redesign  of  main  transmission,  other  gearboxes, 
and  rotor  actuators  to  reduce  survivability  deficiencies 
previously  identified  during  LFT&E. 

2.  Address  H-l  ballistic  survivability  concerns  in  the  future 
when  aircraft  components  are  redesigned  or  replaced.  Place 
particular  emphasis  on  improving  the  run-dry  capabilities 
of  the  main  rotor  transmission  and  combining  gearbox 
housings  following  loss  of  lubricant. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Limit  the  use  of  Brite  Star  II-derived  geographic 
coordinates  for  precision  targeting  until  corrections  are 
identified,  implemented,  and  tested. 

2.  Expand  the  virtual  targeting  reticule’s  indicating  range 
beyond  the  APKWS  launch  envelope  and  provide  an 
indication  when  the  aircraft  is  out  of  the  APKWS  launch 
envelope.  Continue/complete  development  of  the  virtual 
targeting  reticule  implementation  and  address  pilot 
complaints. 

3.  Continue  efforts  to  increase  the  availability  of  spare  parts, 
especially  rotor  system  components,  to  improve  aircraft 
availability. 
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Infrared  Search  and  Track  (IRST) 


Executive  Summary 

•  The  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  issued  a  Milestone  C  Acquisition  Decision 
Memorandum  on  March  24,  2015,  approving  entrance  into  the 
Product  and  Deployment  phase  and  Lot  1  of  Block  I  low-rate 
initial  production  (LRIP).  The  memorandum  directed  the 
program  to  complete  a  second  operational  assessment  (OA  2) 
and  to  develop  mitigation  plans  to  address  the  significant  risks 
to  effectiveness  (identified  in  OA  1,  conducted  in  FY14)  prior 
to  approving  Lot  2  LRIP. 

•  Developmental  testing  progressed  during  FY15,  expanding  the 
flight  envelope  in  which  the  sensor  can  be  employed  through 
aeromechanical  testing,  characterizing  sensor  performance 
(including  testing  algorithm  enhancement  intended  to  address 
problems  identified  in  OA  1),  and  testing  integration  with  the 
F/A-18  weapons  system. 

•  The  problems  identified  in  OA  1,  discussed  in  detail  in 
DOT&E’s  December  2014  classified  OA  1  report,  identified 
areas  of  concern.  Mission-level  operational  testing  is  needed 
to  demonstrate  that  the  Infrared  Search  and  Tracking  (IRST) 
system  will  provide  the  F/A-18  an  effective  combat  capability. 
VX-9  Conducted  OA  2  in  November  2015.  OA  2  will  help 
determine  whether  improvement  seen  in  developmental  testing 
will  translate  to  the  operational  environment.  OA  2  will  help 
inform  the  decision  to  enter  IOT&E  and  will  support  the  Lot  2 
LRIP  decision. 

•  The  Navy  intends  for  the  readiness  review  for  IOT&E  to  occur 
in  March  2016. 

System 

•  The  IRST  system  consists  of  a  passive  long-wave  infrared 
receiver  (IRR),  a  processor,  inertial  measurement  unit  (IMU), 
and  environmental  control  unit  (ECU).  The  IRR,  processor, 
IMU,  and  ECU  are  housed  within  the  Sensor  Assembly 
Structure  (SAS).  The  SAS  attaches  to  the  front  of  the  Fuel 
Tank  Assembly  that  is  mounted  to  the  aircraft  on  the  BRU-32 
bomb  rack.  The  Navy  designed  the  IRST  to  be  flown  on  the 
F/A-18E/F  and  it  will  be  built  into  a  modified  centerline  fuel 
tank. 


•  The  Navy  is  developing  and  fielding  the  system  in  two  blocks: 
Block  1  will  reach  Initial  Operational  Capability  in  FY18  and 
use  components  from  the  F-15K/SG  IRR  that  derive  from 

the  F-14  IRST  system.  Block  II  is  planned  to  begin  after 
the  Block  I  Full-Rate  Production  Decision  Review  and  will 
include  an  improved  IRR  and  updated  processors. 

•  The  Navy  intends  to  produce  a  total  of  170  IRST  systems. 
There  will  be  60  Block  I  systems,  which  will  eventually  be 
updated  to  the  Block  II  configuration;  the  Navy  will  build  an 
additional  110  Block  II  systems. 

Mission 

Commanders  will  use  IRST  in  a  radar-denied  environment  to 
locate  and  destroy  enemy  forces.  The  IRST  system  is  intended 
to  allow  the  F/A-18E/F  to  operate  and  survive  against  existing 
and  emerging  air  threats  by  enhancing  situational  awareness  and 
providing  the  ability  to  acquire  and  engage  targets  beyond  visual 
range. 

Major  Contractors 

•  The  Boeing  Company  -  St  Louis,  Missouri 

•  Lockheed  Martin  -  Orlando,  Florida 


Activity 

•  The  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  held  a  Milestone  C  Decision  Review  on 
December  2,  2014,  and  issued  a  Milestone  C  Acquisition 
Decision  Memorandum  on  March  24,  2015,  approving 
entrance  into  the  Product  and  Deployment  phase.  The 
memorandum  approved  Lot  1  (6  systems)  of  Block  1  LRIP, 
but  directed  the  program  to  complete  a  second  OA  (OA  2)  and 
develop  mitigation  plans  to  address  the  significant  risks  to 


effectiveness  identified  in  OA  1  (conducted  in  FY14)  prior  to  a 
decision  to  award  Lot  2(12  systems)  of  LRIP. 

•  The  program  entered  Integrated  Test  phase  IT-C1  in 

December  2014  following  the  December  Milestone  C  review. 
The  objectives  of  this  test  phase  are  to  clear  the  flight  envelope 
in  which  the  system  can  be  employed,  characterize  sensor 
performance  (including  testing  algorithm  enhancements 
intended  to  address  problems  identified  in  OA  1),  and  test 
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integration  of  IRST  with  the  F/A-18  weapons  system.  The 
Naval  Air  Systems  Command  (NAVA1R)  IT-C1  report,  in 
conjunction  with  the  Commander,  Operational  Test  and 
Evaluation  Force  OA  2  report,  will  support  the  program’s 
decision  review  on  whether  to  enter  IOT&E. 

•  Based  on  the  results  of  aeromechanical  testing  conducted 
by  VX-23  at  Patuxent  River,  Maryland,  NAVA1R  issued  a 
flight  clearance  in  May  2015  that  allowed  flight  test  with 
the  full  envelope  of  flight  conditions  when  the  fuel  tank 
is  empty.  NAVA1R  issued  another  flight  clearance  in  July 
permitting  flight  with  fuel  in  the  tank,  but  under  restricted 
conditions.  Carrier  qualification  testing  is  expected  to  be 
completed  in  December  2015,  which  will  be  followed  by 
further  aeromechanical  testing  required  to  clear  the  full  flight 
envelope,  including  fuel  carriage. 

•  VX-31  at  China  Lake,  California,  conducted  performance 
flight  testing,  providing  data  needed  to  verify  specification 
compliance  and  to  certify  the  system’s  readiness  to  transition 
to  IOT&E.  VX-31  has  tested  eight  new  releases  of  IRST 
software  since  OA  1.  These  releases  include  algorithm 
improvements  intended  to  correct  problems  seen  in  OA  1 . 
Expansion  of  the  available  flight  envelope  has  permitted 
testing  in  more  dynamic  conditions  and  allowed  participation 
by  VX-31  in  a  large  force  exercise  at  Nellis  AFB,  Nevada,  in 
June  2015. 

•  VX-3 1  conducted  integration  of  the  IRST  system  with 
the  weapon  system,  including  the  ability  to  fuse  IRST 
sensor  information  with  other  sensors  (such  as  radar)  into 
Multi-Sensor  Integration  (MSI)  tracks  on  which  weapons 
can  be  employed.  A  progression  of  simulated  AIM- 120  shots 
using  captive  carry  missiles  have  been  performed  with  more 
captive  carry  missions  expected  prior  to  a  live  AIM- 120  shot 
against  a  QF-4  drone  in  3QFY16  or  4QFY16. 

•  VX-9  conducted  OA  2  in  November  2015.  The  OA  will 
include  four  simulated  combat  trials  that  will  provide  data 
to  evaluate  the  ability  of  the  system  to  support  detection, 
tracking,  and  missile  employment  in  a  dynamic,  operationally 
representative  environment.  OA  2  should  help  inform  the 
decision  to  enter  IOT&E  as  well  as  the  Lot  2  LRIP  decision. 

•  In  July  2015,  the  Navy  requested  USD(AT&L)  designate 
IRST  as  an  Acquisition  Category  I  (ACAT  I)  program  (from 
ACAT  II)  because  the  program  has  exceeded  the  research  and 
development  dollar  threshold  due  to  the  research  development 
test  and  evaluation  funding  provided  in  the  FY16  presidential 
budget  to  Congress  for  Block  II  development.  The  Navy 
also  requested  delegation  of  decision  authority  to  the  Navy 
(i.e.,  designation  as  an  ACAT  IC).  Designation  to  ACAT  IC  is 
expected  in  November  2015. 

•  A  Lot  2  LRIP  decision  is  expected  in  December  2015  and  the 
IOT&E  readiness  review  is  currently  planned  for  March  2016. 

Assessment 

•  The  system  tested  in  OA  1  could  not  detect  and  track 
targets  well  enough  to  support  weapons  employment  in  an 
environment  that  reflects  realistic  fighter  employment  and 
tactics. 


•  The  Key  Performance  Parameter  (KPP)  and  the  derived 
contract  specification  for  detection  and  tracking  describe  only 
a  narrow  subset  of  the  operational  environments  where  the 
Navy  will  employ  IRST.  Meeting  the  KPP  (with  a  narrow 
reading  of  the  KPP  requirement)  does  not  ensure  a  useful 
combat  capability. 

•  The  program  has  made  an  effort  to  develop  a  more  robust 
tracking  capability  to  provide  capability  outside  the  KPP 
conditions.  This  effort  includes  the  introduction  of  a  new 
tracking  algorithm  and  the  release  and  the  testing  of  eight  new 
IRST  software  versions  since  OA  1 . 

•  While  improvement  has  been  seen  in  developmental  testing, 
the  nature  of  the  problems  with  detection  and  tracking 
identified  in  OA  1  are  such  that  mission-level  operational 
testing  is  needed  to  demonstrate  that  these  problems  will 
not  prevent  IRST  from  providing  the  F/A-18  an  effective 
combat  capability.  Until  satisfactory  performance  has  been 
demonstrated  in  an  operationally-representative  environment 
that  includes  jamming  and  dynamic  maneuvering,  the  risk  of 
an  unsuccessful  IOT&E  is  high. 

•  The  Navy’s  decision  to  add  a  second  OA  that  includes  realistic 
mission  scenarios  prior  to  IOT&E  will  significantly  reduce 
the  risk  of  an  unsuccessful  IOT&E  by  providing  test  data  in  a 
dynamic  maneuvering  and  jamming  environment  prior  to  the 
decision  to  enter  IOT&E. 

•  The  program  has  decided  to  use  a  mechanical  boresight 
procedure  instead  of  using  the  line-of-sight  estimation 
algorithm  known  as  servo-transfer  alignment  as  originally 
planned.  The  Navy  is  assessing  the  impact  on  IRST  logistics. 
The  system’s  logistics  will  be  affected  by  how  long  the 
boresight  will  hold  and  the  support  equipment  that  will  be 
required.  The  contractor  has  reported  that  ground  vibration 
testing  shows  the  boresight  should  hold  longer  than  three 
times  the  mean  time  between  repairs.  The  program  is  tracking 
angular  accuracy  during  flight  test  to  look  for  any  degradation. 
The  maintenance  task  analysis  for  the  intermediate-level 
(I-level)  (aboard  ship)  and  depot-level  (D-level)  to  determine 
the  maintenance  tasks  for  each  level  is  currently  in  process. 
However,  based  on  the  types  of  maintenance  that  will  likely 
require  realignment,  I-level  (aboard  ship)  and  depot-level 
support  equipment  for  alignment  will  may  be  needed  for  the  I 
and/or  D-levels. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  should 
continue  to  address  the  two  FY14  recommendations: 

1 .  Explicitly  state  detection  and  tracking  requirements  for  the 
range  of  operational  conditions  in  which  the  Navy  expects 
to  employ  the  system.  N98,  which  has  responsibility 

for  naval  aviation  warfighting  requirements,  is  currently 
authoring  a  requirements  clarification  memorandum 
regarding  the  KPP  scenario. 

2.  Improve  detection  and  tracking  performance  prior  to  entry 
into  IOT&E.  Improvements  will  be  tested  in  OA  2  in 
November  2015. 

•  FY15  Recommendations.  None. 
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Integrated  Defensive  Electronic  Countermeasures 

(IDECM) 


Executive  Summary 

•  The  Navy  ended  Integrated  Defensive  Electronic 
Countermeasures  (IDECM)  Block  IV  (IB-4)  hardware  testing 
on  November  17,  2015.  The  Navy  did  not  conduct  FOT&E  in 
accordance  with  the  original  DOT&E -approved  test  plan  due 
to  missed  test  points;  however,  the  FOT&E  that  the  Navy  did 
accomplish  is  adequate  for  an  operational  assessment  (OA)  for 
the  subsequent  Software  Improvement  (SWIP)  testing.  The 
Navy,  in  collaboration  with  DOT&E,  began  conversion  of  the 
FOT&E  to  an  OA  in  August  2015  since  the  Navy’s  intent  is  to 
field  IB-4  hardware  with  mature  SWIP  software  as  the  Navy’s 
final  configuration  to  counter  modern  threats. 

•  Preliminary  analysis  indicates  that  IB-4  hardware  with 
precursor  SWIP  software  is  as  effective  on  the  F-18  E/F 
platform  as  the  currently  fielded  IB-3  system. 

•  On  May  26,  2015,  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition  issued  an  Acquisition 
Decision  Memorandum  approving  the  FY15  buy  for  IB-4 
hardware.  In  addition,  the  Deputy  Assistant  Secretary  of 

the  Navy  for  Air  Programs  issued  a  Program  Deviation 
Report  to  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development,  and  Acquisition  authorizing  the  Navy  to  equip 
three  F-18  E/F  squadrons  with  IB-4  hardware  with  precursor 
SWIP  software  as  an  early  fielding  step  towards  achieving 
Initial  Operational  Capability  for  the  F-18  E/F  fleet,  and  use 
data  from  the  early  fielded  systems  to  enhance  the  reliability 
growth  plan. 

•  DOT&E  will  submit  a  classified  IB-4  OA  report  in  early  2016 
assessing  preliminary  system  operational  effectiveness  and 
suitability. 

System 

•  The  IDECM  system  is  a  radio  frequency,  self-protection 
electronic  countermeasure  suite  on  F/A-18  aircraft.  The 
system  is  comprised  of  onboard  and  off-board  components. 
The  onboard  components  receive  and  process  radar 
signals  and  can  employ  onboard  and/or  off-board  jamming 
components  in  response  to  identified  threats. 

•  There  are  four  IDECM  variants:  Block  I  (IB- 1),  Block 
II  (IB-2),  Block  III  (IB-3),  and  Block  IV  (IB-4).  All  four 
variants  include  an  onboard  radio  frequency  receiver  and 
jammer. 

-  IB-1  (fielded  FY02)  combined  the  legacy  onboard 
receiver/jammer  (ALQ-165)  with  the  legacy  (ALE-50) 
off-board  towed  decoy. 

-  IB-2  (fielded  FY04)  combined  an  improved  onboard 
receiver/jammer  (ALQ-214)  with  the  legacy  (ALE-50) 
off-board  towed  decoy. 


-  IB-3  (fielded  FY11)  combined  the  improved  onboard 
receiver/jammer  (ALQ-214)  with  a  new  (ALE-55) 
off-board  fiber-optic  towed  decoy  that  is  more  integrated 
with  the  ALQ-214. 

-  IB-4  with  Software  Improvement  Program  (SWIP) 
(currently  in  test)  replaces  the  onboard  receiver/jammer 
(ALQ-214(V)3)  with  a  lightweight,  repackaged  onboard 
jammer  (ALQ-214(V)4  and  ALQ-214(V)5).  IB-4 

also  replaces  the  ALQ-126B  to  provide  advanced, 
carrier-capable  jamming  to  the  F/A-18C/D  for  the  first 
time.  IB-4  (without  SWIP)  fielded  to  three  squadrons  in 
FY15. 

•  The  additional  program  to  provide  IB-4  the  capability  to  either 
deny-delay  targeting  of  the  F/A-18  by  enemy  radars,  known 
as  the  SWIP,  is  in  early  development  with  developmental  test 
flights  planned  to  begin  in  November  2015.  The  intent  of 
SWIP  is  to  allow  IB-4  hardware  to  counter  modern  threats. 

•  The  F/A-18E/F  installation  includes  off-board  towed  decoys. 
The  F/A-18C/D  installation  includes  only  the  onboard 
receiver/jammer  components  and  not  the  towed  decoy. 

Mission 

•  Combatant  Commanders  will  use  IDECM  to  improve 
the  survivability  of  Navy  F/A-18  strike  aircraft  against 
radio  frequency-guided  threats  while  flying  air-to-air  and 
air-to-ground  missions. 

•  The  Navy  intends  to  use  IB-3’s  and  lB-4’s  complex  jamming 
capabilities  to  increase  survivability  against  modern 
radar-guided  threats. 

Major  Contractors 

•  ALE-55:  BAE  Systems  -  Nashua,  New  Hampshire 

•  ALQ-214:  Harris  -  Clifton,  New  Jersey 

•  ALE-50:  Raytheon  Electronic  Warfare  Systems  -  Goleta, 
California 
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Activity 

IB-4 

•  The  Navy  decided  in  April  20 12  to  transition  production 
of  the  IB-3  systems  to  IB-4  systems  (production  lot  buy 
9).  The  IB-4  system  has  been  in  full-rate  production  as  of 
production  lot  buy  11  and  the  Navy  no  longer  procures  IB-3 
systems. 

•  The  IB-4  FOT&E  experienced  delays  for  several  reasons. 
Problems  with  the  Environmental  Control  System  cooling 
and  cabin  pressure  on  test  F/A-18  C/D  aircraft  with  IB-4 
hardware  installed,  as  well  as  additional  test  aircraft 
unscheduled  and  phase  maintenance,  residted  in  test  aircraft 
being  unavailable.  Lack  of  test  aircraft  led  to  missed  range 
periods  at  Nevada  Test  and  Training  (NTTR)  and  Electronic 
Combat  Range  (ECR).  Another  problem  was  unavailability 
of  the  China  Lake  ECR  test  range  due  to  both  scheduled 
and  unscheduled  maintenance  of  range  threats. 

•  Due  to  the  delays  to  IB-4  testing  and  the  Navy’s  decision 
to  no  longer  procure  IB-3  systems,  new  F/A-18E/F  aircraft 
would  not  include  installed  jammers.  This  resulted  in  the 
Navy’s  decision  to  field  IB-4  with  precursor  SWIP  software 
to  three  squadrons  before  the  completion  of  FOT&E. 

•  On  May  26,  2015,  the  Assistant  Secretary  of  the  Navy 
for  Research,  Development,  and  Acquisition  issued  an 
Acquisition  Decision  Memorandum  approving  the  FY15 
buy  for  IB-4  hardware.  In  addition,  the  Deputy  Assistant 
Secretary  of  the  Navy  for  Air  Programs  issued  a  Program 
Deviation  Report  to  the  Assistant  Secretary  of  the  Navy 
for  Research,  Development,  and  Acquisition  authorizing 
the  Navy  to  equip  three  F-18  E/F  squadrons  with  IB-4 
hardware  with  precursor  SWIP  software  as  an  early  fielding 
step  towards  achieving  Initial  Operational  Capability  for 
the  F-18  E/F  fleet,  and  use  data  from  the  fielded  systems  to 
support  the  reliability  growth  program. 

•  In  late  June  2015,  DOT&E  recommended  the  Navy 
convert  the  planned  IB-4  FOT&E  to  an  OA  to  facilitate 
SWIP  development  and  support  the  SWIP  FOT&E. 

DOT&E  and  the  Navy  pursued  this  approach  since  the 
final  IDECM  system  expected  to  be  fielded  throughout 
the  fleet  is  the  IB-4  hardware  with  the  SWIP  software  and 
the  September  30,  2015  expiration  of  FY14  research  and 
development  funding  for  IB-4  testing.  The  Navy  began 
FOT&E  conversion  to  an  OA  in  August  2015  and  will 
continue  IB-4  testing  under  SWIP  operational  testing. 

•  On  November  17,  2015,  the  Navy  declared  end  of  test  with 
the  following  completed  on  IB-4  hardware: 

-  89  of  1 60  planned  test  conditions  at  the  ECR  at  China 
Lake,  California,  with  18  of  80  on  the  C/D  and  71  of  80  on 
the  E/F. 

-  Sufficient  test  at  NTTR  to  assess  preliminary  IDECM 
performance. 

-  All  planned  laboratory  testing,  including  a  dense  emitter 
scenario  and  closed-loop  hardware-in-the-loop  testing. 

-  A  limited  maintenance  demonstration. 


•  Failure  scoring  boards  for  the  OA  flights  in  support  of 
assessing  system  reliability  have  not  convened,  but  are 
scheduled  for  November  2015. 

•  DOT&E  will  submit  a  classified  OA  report  detailing  the 
results  of  IB-4  testing  in  early  2016. 

SWIP 

•  The  Navy  completed  the  first  major  test  event  for  the 
SWIP  program  in  July  2015  at  a  systems  integration  lab  at 
Edwards  AFB,  California,  against  a  surrogate  threat  system. 

•  The  Navy  conducted  an  IDECM  SWIP  Flight  Readiness 
Review  on  November  4,  2015.  The  Navy  anticipates 
beginning  IB-4  hardware  regression  flight  testing  with 
SWIP  software  in  2QFY16  at  NTTR  and  ECR. 

Assessment 

IB-4 

•  DOT&E  discovered  suitability  problems  with  IB-4  on 
the  F/A-18C/D  platform  during  integrated  testing  and 
confirmed  them  during  FOT&E. 

-  Cabin  pressure  problems  and  avionics  cooling  air  degrades 
were  observed  at  about  20,000  feet  in  altitude,  which 
delayed  FOT&E. 

-  Cooling  problems  were  observed  with  the  Environmental 
Control  System.  The  Navy  made  repairs  to  the  aircraft 
under  test;  however,  until  they  have  also  completed  these 
repairs  to  the  Environmental  Control  System  on  legacy 
IB-4  aircraft,  DOT&E  assesses  it  is  likely  not  suitable  as 
integrated  on  the  F/A-18C/D  platform. 

•  The  Navy  corrected  the  deficiency  caused  by  interaction 
between  the  ALR-67(V)2  and  (V)3  radar-warning  receivers 
and  IB-4  system,  which  caused  false  threat  symbols  to 

be  displayed.  However,  the  Navy  deferred  correcting 
the  deficiency  in  which  the  APG-79  radar  is  falsely 
identified  to  the  ALQ-2 14(V)4  by  the  ALR-67(V)2  and 
(V)3  radar-warning  receivers  to  a  wingman  compatibility 
working  group  composed  of  multiple  Program  Offices. 

•  IB-4  testing  revealed  an  effectiveness  problem  against  an 
operational  threat  for  the  F/A-18C/D,  which  would  decrease 
overall  survivability  for  the  aircraft  if  not  corrected. 
Previous  testing  using  developmental  test  aircraft  and 

a  previous  software  version,  and  not  flying  operational 
maneuvers,  did  not  reveal  an  effectiveness  problem.  Further 
details  on  this  problem  are  classified. 

•  The  planned  IB-4  FOT&E  was  not  completed  in  accordance 
with  its  DOT&E-approved  test  plan  due  to  missed  test 
points.  However,  testing  to  date  is  adequate  since  the  IB-4 
FOT&E  conversion  to  an  OA  supports  the  SWIP  program. 

SWIP 

•  While  the  OA  supports  the  SWIP  program,  the  missed 
IB-4  test  points  must  be  collected  during  SWIP  integrated 
regression  testing  to  ensure  adequate  data  for  analysis  to 
determine  performance. 
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•  Preliminary  analysis  indicates  the  IB-4  hardware  with 
precursor  SWIP  software,  which  the  Navy  chose  for  their 
three  early  fielding  F-18  E/F  squadrons,  is  as  effective  as 
the  currently  fielded  IB-3  system  on  the  E/F  platform. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  addressed 

some  previous  recommendations;  however,  the  following 

remain  outstanding: 

IDECM  System 

1.  The  Navy  should  reorganize  complex  IDECM  software  to 
minimize  potential  problems  that  could  occur  during  the 
IDECM  SWIP. 

2.  The  Navy  should  develop  hardware  and/or  software 
changes  to  provide  pilots  with  correct  indications  of 
whether  a  decoy  was  completely  severed. 

3.  The  Navy  should  investigate  the  effects  of  IDECM  on  threat 
missile  fuses. 

4.  The  Navy  should  use  the  high-fidelity,  accredited 
F/A-18  radar  cross  section  (RCS)  data  when  accomplishing 
analysis  and  hardware-in-the-loop  testing,  and  ensure  that 
the  RCS  models  account  for  the  entire  F/A-18  airframe 
configuration. 

Electronic  Warfare  Warfighting  Improvements 

5.  In  coordination  with  the  Defense  Intelligence  Agency,  the 
Navy  should  update  the  warhead  probability  of  kill  data 


in  requirements  documents  to  confirm  IDECM  effects  are 
sufficient  to  enhance  aircraft  survivability. 

FY15  Recommendations.  The  Navy  should: 

1 .  Improve  data  collection  processes  to  allow  for  an  adequate 
collection  of  suitability  data  during  the  SWIP/1B-4 
operational  test  period. 

2.  Use  the  results  from  IB-4  testing  accomplished  to  date  to 
prioritize  system  shortfall  resolution  for  the  SWIP  FOT&E. 

3.  The  Navy  should  ensure  that  all  resources  needed  for  the 
SWIP  FOT&E  will  be  available  by  the  start  of  testing  in 
1QFY17. 

4.  The  Navy  should  ensure  that  the  ALR-67(V)3 
radar-warning  receiver  interface  with  IDECM  is  updated 
so  that  aircrew  have  accurate  situational  awareness  of  the 
effectiveness  of  SWIP  deny-delay  countermeasures. 

5.  In  addition  to  the  previous  recommendation  that  the  Navy 
should  investigate  the  effects  of  IDECM  on  threat  missile 
fuses,  the  Navy  should  include  warhead  fusing  in  this 
investigation. 

6.  During  SWIP  integrated  regression  testing,  the  Navy  should 
collect  the  missed  IB-4  FOT&E  test  points. 
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Joint  High  Speed  Vessel  (JHSV) 
(Expeditionary  Fast  Transport) 


Executive  Summary 

•  Due  to  the  unavailability  of  the  Mobile  Landing  Platform  with 
the  Core  Capability  Set  (MLP  (CCS))  and  the  U.S.  Navy’s 
Sea,  Air,  Land  Team  (SEAL)  Delivery  Vehicle  (SDV)  during 
the  FY13-14  10T&E,  and  to  further  examine  suitability,  the 
Navy  conducted  an  FOT&E  on  the  Joint  High  Speed  Vessel 
(JHSV)  (now  called  Expeditionary  Fast  Transport)  in  FY14 
and  FY15. 

•  The  Commander,  Operational  Test  and  Evaluation  Force, 
in  conjunction  with  Marine  Corps  Operational  Test  and 
Evaluation  Activity,  conducted  three  FOT&E  events:  the  first 
two  in  June  and  October  2014,  and  the  third  in  April  2015. 

The  first  two  events  consisted  of  mooring  operations  with 
the  MLP  (CCS).  The  third  event  consisted  of  launching  and 
retrieving  the  SDV  from  the  JHSV  while  at  sea. 

•  On  September  22,  2015,  DOT&E  submitted  an  FOT&E  report 
and  found  the  following: 

-  JHSV  interoperability  with  MLP  (CCS)  is  not 
operationally  effective  since,  by  design  (ramp  limitation), 

it  can  only  conduct  vehicle  transfers  when  conducted  in  sea 
states  with  significant  wave  heights  of  less  than  0. 1  meters 
(approximates  a  Sea  State  1),  which  are  normally  found 
only  in  protected  harbors.  The  JHSV  is  operationally 
effective  at  launch  and  recovery  of  the  SDV. 

-  Although  JHSV  testing  continues  to  show  the  ship  is 
operationally  suitable  in  terms  of  minimum  availability,  the 
Ship  Service  Diesel  Generators  (SSDGs),  waterjets,  and 
Ride  Control  System  (RCS)  did  not  meet  their  individual 
reliability  goals. 

-  The  operational  restriction  of  the  JHSV’s  Safe  Operating 
Envelope  (SOE)  is  a  major  limitation  of  the  ship  class  that 
must  be  factored  into  all  missions. 

System 

•  The  JHSV  is  a  high-speed,  shallow-draft  surface  vessel 
designed  for  intra-theater  transport  of  personnel  and  medium 
cargo  payloads  for  the  joint  force. 

•  JHSV  bridges  the  gap  between  large-capacity,  low-speed 
sealift  and  small-capacity,  high-speed  airlift. 

•  JHSV  is  a  redesign  of  a  commercial  catamaran  capable 
of  accessing  relatively  austere  ports.  Classified  as  a 
non-combatant  vehicle,  JHSV  has  limited  self-protection 
capability.  Design  characteristics  include: 

-  Propulsion  provided  by  four,  diesel  engine-powered  water 
jets 

-  At-sea  refueling  capability 


-  Support  for  3 12  embarked  troops  for  up  to  96  hours  or 
104  troops  for  14  days 

-  An  integrated  ramp  capable  of  loading/off-loading  military 
vehicles,  including  combat-loaded  main  battle  tanks 

-  A  flight  deck  with  helicopter  refueling  capability 

Mission 

Combatant  Commanders  will  use  the  JHSV  to  support  the 
flexible,  agile  maneuver  and  sustainment  of  combat  forces 
between  forward  operating  bases,  ports,  austere  littoral  access 
points,  and  the  sea  base.  Combatant  Commanders  may  employ 
the  JHSV  in  a  transport/resupply  role  in  benign,  non-hostile 
environments  to: 

•  Rapidly  transport  medium  payloads  of  Marine  Corps  or  Army 
cargo  and  combat-ready  troops  over  intra-theatre  distances 
between  shore  nodes 

•  Deliver  troops,  combat-loaded  vehicles,  and  equipment  ready 
to  be  employed,  using  only  ports  with  pier  or  quay  wall  access 
and  no  other  infrastructure 

•  Support  sustainment  of  forces  between  forward  operating 
bases,  ports,  and  austere  littoral  access  points  that  would  be 
prohibitive  for  larger  ships  to  access 

Major  Contractor 

Austal  USA  -  Mobile,  Alabama 
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Activity 

•  From  June  2014  through  April  2015,  the  Navy’s  Commander, 
Operational  Test  and  Evaluation  Force  and  Marine  Corps 
Operational  Test  and  Evaluation  Activity  conducted  an 
FOT&E  on  USNS  Spearhead  (JHSV  1)  and  USNS  Millinocket 
(JF1SV  3)  during  the  following  test  periods: 

-  Two  separate  events,  one  in  June  in  the  Long  Beach  harbor 
in  California,  and  the  other  in  October  2014  at  sea  off  the 
coast  of  Camp  Pendleton,  California,  were  conducted  to 
test  the  JHSV/MLP  (CCS)  interface. 

-  Reliability,  availability,  and  maintainability  (suitability) 
data  were  collected  during  all  underway  periods  of  the 
USNS  Spearhead  from  June  2014  through  June  2015 
during  its  transit  and  deployment  to  the  6th  Fleet  in  the 
Mediterranean  Sea  and  off  the  coast  of  West  Africa. 

-  The  launch  and  recovery  of  the  SDV  from  the  JHSV  while 
inside  and  outside  Pearl  Harbor,  Hawaii,  in  April  2015. 
Testing  did  not  include  an  evaluation  of  JHSV’s  ability  to 
host  a  Special  Operations  Force  (SOF)  mission  package. 

•  In  September  2015,  DOT&E  submitted  an  FOT&E  report 
detailing  the  results  of  testing. 

•  The  Navy  and  the  Marine  Corps  conducted  all  testing  in 
accordance  with  a  DOT&E  approved  test  plan. 

Assessment 

•  JHSV  is  not  operationally  effective  interfacing  with  MLP 
(CCS)  for  open-ocean,  at-sea  transfer  of  vehicles.  The 
JHSV  ramp  cannot  handle  the  small,  but  continual,  relative 
movement  of  the  two  ships  when  moored  skin-to-skin. 
Although  vehicles  were  successfully  transferred  inside  a 
protected  harbor,  transfer  operations  at-sea  failed. 

•  JHSV  is  operationally  effective  at  launching  and  recovering 
the  SDV  through  Sea  State  3  (significant  wave  height  up  to 
1.25  meters)  although  two  issues  arose  during  testing: 

-  Pendulum  motion  of  the  SDV  when  lifted  by  the  crane 
interfered  with  its  recovery.  Personnel  handling  tending 
lines  were  challenged  with  controlling  the  swinging  of 
the  SDV  as  they  were  returning  the  vessel  to  its  trailer. 
Anti-pendulation  systems  for  cranes  are  becoming 
commercially  available,  and  they  may  help  control  this 
problem. 

-  Waterborne  Navy  SOF  personnel  involved  with  the  launch 
and  recovery  of  the  SDV  reported  high  levels  of  exhaust 
gasses  in  the  vicinity  of  the  launch.  These  gasses  may 
have  an  effect  on  SOF  personnel  readying  themselves  for 
missions  requiring  oxygen  transits. 

•  JHSV  remains  operationally  suitable  although  its  availability 
has  decreased  from  an  estimated  98  percent  reported  at  IOT&E 
to  87  percent  when  including  FOT&E  data.  Main  drivers  of 
ship’s  unavailability  were  the  SSDGs,  waterjets,  and  the  RCS. 

-  The  SSDGs  installed  in  JHSV  demonstrated  poor  reliability 
during  both  IOT&E  and  FOT&E  test  periods.  Their  target 
Mean  Time  Between  Failure  was  8,369  hours,  but  was 
measured  to  be  only  208  hours  in  IOT&E  and  1,563  hours 
during  FOT&E. 


-  The  JHSV  wateijets  demonstrated  poor  reliability  during 
the  first  ship’s  deployment.  All  four  waterjets  suffered 
broken  or  failing  reversing  plates. 

-  The  RCS  internal  mechanism  for  the  forward  foils  has 
failed  repeatedly.  RCS  provides  active  pitch/roll  damping 
to  not  only  smooth  out  the  ride,  but  to  limit  structural 
loading  on  the  ship  bow. 

-  The  bow  structure  USNS  Spearhead  was  damaged  during 
her  deployment  due  to  sea  slam  events  in  higher  sea  state 
conditions.  Because  of  this,  the  Navy  is  reinforcing  the 
bows  on  all  JHSVs  under  construction  and  back-fitting  the 
reinforcement  on  hulls  1  through  4.  The  reinforcement 
of  the  bow  structure  does  not  expand  the  SOE,  but  should 
allow  full  use  of  the  ship,  within  the  original  SOE,  without 
continued  risk  of  damage.  The  operational  restriction  of 
the  SOE  is  a  major  limitation  of  the  ship  class  that  must  be 
factored  into  all  missions.  To  utilize  the  speed  capability  of 
the  ship,  seas  must  not  exceed  Sea  State  3  (significant  wave 
height  up  to  1.25  meters).  At  Sea  State  4  (significant  wave 
height  up  to  2.5  meters),  the  ship  must  slow  to  15  knots. 

At  Sea  State  5  (significant  wave  height  up  to  4  meters),  the 
ship  must  slow  to  5  knots.  Above  Sea  State  5,  the  ship  can 
only  hold  position  and  await  calmer  seas. 

Recommendations: 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 

progress  in  addressing  previous  recommendations  from  both 

FY13  and  FY14;  however,  several  recommendations  remain 

outstanding.  The  Navy  still  needs  to: 

1 .  Determine  the  best  self-deployed  transit  speed  to  achieve 
the  4,700-nautical  mile  un-refueled  range  requirement. 

2.  Determine  a  transit  speed  that  allows  for  a  600  short  ton 
load  delivery  to  1,200  nautical  miles. 

3.  Determine  an  outfitted  weight  for  each  hull  to  enable 
mission  planners  to  characterize  fully  loaded  transit 
capability. 

4.  Evaluate  design  improvements  identified  during  the  Total 
Ship  Survivability  Trials  and  implement  those  that  will 
enhance  the  ship’s  survivability. 

5.  Demonstrate  11 -meter  Rigid  Hull  Inflatable  Boat  launch 
capability  in  Sea  State  3  (wave  heights  up  to  4  feet). 

6.  Review  and  modify  tactics,  techniques,  and  procedures 
to  safely  launch  Rigid  Hull  Inflatable  Boats  in  sea  states 
greater  than  Sea  State  2. 

7.  Consider  a  replacement  for  the  Cargo  Loading  Trailer  if 
a  JHSV  is  utilized  routinely  to  transport  20-foot  storage 
containers. 

8.  Implement  a  reliability  growth  program  for  the  SSDGs. 

9.  Resolve  and  retest  the  significant  cybersecurity 
vulnerabilities  identified  in  the  classified  DOT&E  combined 
IOT &E  and  LFT &E  report. 

10.  Provide  safety  lanyards  and  harnesses  for  embarked  security 
team  members  that  man  gun  mounts.  Additionally,  provide 


214 


JHSV 


F Y1  5  NAVY  PROGRAMS 


hands-free  communication  devices  to  help  coordinate  firing 
engagements. 

11.  Investigate  the  casualty  problem  with  JHSV’s  ramp  that 
occurred  during  the  interface  test  with  MLP  (CCS)  in 
October  2014.  If  necessary,  reevaluate  the  need  for  at  sea 
skin-to-skin  operations  between  JHSV  and  MLP  (CCS). 

•  FY15  Recommendations.  The  Navy  should  address  the 

following  recommendations  from  the  September  2015  FOT&E 
report: 

1.  Modify  the  JHSV  ramp  to  increase  its  sea  state  rating,  or 
develop  a  new,  higher  sea  state  rated  ramp,  then  retest  at-sea 
equipment  transfers  with  MLP  (CCS)  in  order  to  conduct 
open  ocean  equipment  transfers  between  JHSV  and  MLP 
(CCS). 

2.  Investigate  the  availability  of  a  pendulation  control  system 
for  the  JHSV  stern-mounted  crane. 


3.  Evaluate  the  effect  of  JHSV  exhaust  gases  on  SOF 
personnel  readying  themselves  for  oxygen  transits. 

4.  Evaluate  JHSV  capabilities  to  support  personnel  and 
equipment  for  various  SOF  mission  packages. 

5.  Evaluate  whether  repairs  and  alterations  to  the  waterjet 
reversing  buckets,  along  with  alterations  to  the  ship’s 
autopilot  system,  resolve  the  failure  mode  of  this 
equipment,  or,  alternately,  investigate  a  replacement 
schedule  to  minimize  waterjet  casualty  downtime. 

6.  Evaluate  whether  the  repairs  and  alterations  to  the  internal 
operating  mechanism  of  the  forward  ride  control  foils 
resolves  the  failure  mode. 

7.  Complete  structural  reinforcement  of  bow  structure  on  the 
class. 
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Joint  Standoff  Weapon  (JSOW) 


Executive  Summary 

•  The  Navy  began  operational  testing  of  the  Joint  Standoff 
Weapon  (JSOW)  C-l  in  April  2015.  Problems  identified 
during  FY12-13  integrated  testing  resulted  in  follow-on 
integrated  testing  in  late  FY14  and  pushed  operational  testing 
to  FY15.  The  JSOW  C-l  operational  testing  is  scheduled  to 
complete  in  FY16. 

•  Preliminary  results  to  date  indicate: 

-  Weapon  impact  accuracy  for  moving  maritime  targets  is 
well  within  the  accuracy  requirement  value,  and  accuracy 
performance  against  stationary  land  targets  has  been 
maintained. 

-  The  JSOW  C-l  Mean  Flight  Hours  Between  Operational 
Mission  Failure  (MFHBOMF)  remains  below  the 
requirement  value,  primarily  the  result  of  software-driven 
problems,  but  is  showing  progress  towards  meeting  the 
requirement  value  by  the  end  of  operational  testing. 
Achieving  adequate  assessment  of  MFHBOMF  during 
operational  testing  is  an  area  of  moderate  risk. 

-  The  pilot-vehicle  interface  (PVI)  has  improved,  but  there 
remain  some  minor  challenges  the  aircrew  must  work 
around  for  successful  mission  execution.  The  Navy  is 
incorporating  fixes  to  address  this  into  the  F/A-18E/F  H12 
Operational  Flight  Program  (OFP),  scheduled  for  release  in 
FY17. 

•  Testing  of  the  implemented  updates  to  the  JSOW  software  to 
address  these  problems  validate  the  use  of  developmental  and 
integrated  test  data  for  DOT&E’s  operational  evaluation  of 
JSOW  C-l. 

System 

•  The  AGM-154  JSOW  family  uses  a  common  and  modular 
weapon  body  capable  of  carrying  various  payloads.  The 
JSOW  is  a  1,000-pound  class,  air-to-surface  glide  bomb 
intended  to  provide  low  observable,  standoff  precision 
engagement  with  launch  and  leave  capability.  All  variants 
employ  a  tightly  coupled  GPS/Inertial  Navigation  System. 

•  AGM-154  A  (JSOW  A)  payload  consists  of  145  BLU-97/B 
combined  effects  submunitions. 

•  AGM-154C  (JSOW  C)  utilizes  an  imaging  infrared  seeker 
and  its  payload  consists  of  an  augmenting  charge  and  follow 
through  bomb  that  can  be  set  to  detonate  both  warheads 
simultaneously  or  sequentially. 


•  AGM-154  A  and  AGM-154C  are  fielded  weapons  and  no 
longer  under  DOT&E  oversight.  AGM-154C-1  (JSOW  C-l) 
adds  moving  maritime  target  capability  and  the  two-way  strike 
common  weapon  datalink  to  the  baseline  AGM-154C  weapon. 

Mission 

•  Combatant  Commanders  use  JSOW  A  to  conduct  pre-planned 
attacks  on  soft  point  and  area  targets  such  as  air  defense  sites, 
parked  aircraft,  airfield  and  port  facilities,  command  and 
control  antennas,  stationary  light  vehicles,  trucks,  artillery,  and 
refinery  components. 

•  Combatant  Commanders  use  JSOW  C  to  conduct  pre-planned 
attacks  on  point  targets  vulnerable  to  blast  and  fragmentation 
effects  and  point  targets  vulnerable  to  penetration  such  as 
industrial  facilities,  logistical  systems,  and  hardened  facilities. 

•  Units  will  use  JSOW  C-l  to  conduct  attacks  against  moving 
maritime  targets  and  have  the  ability  to  retarget  weapons  post 
launch.  JSOW  C-l  will  retain  the  JSOW  C  legacy  capability 
against  stationary  land  targets. 

Major  Contractor 

Raytheon  Company,  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  Operational  testing  began  in  April  2015,  with  the  first  captive  •  During  June  and  July  2015,  the  Navy  completed  six  low 
carry  flight  test  in  June  2015.  humidity  Stationary  Land  Target  captive  flight  test  (CFT) 
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events  at  China  Lake,  California,  with  22  runs  collected. 

•  During  August  and  September  2015,  the  Navy  conducted  high 
humidity  testing  at  Naval  Air  Station  Key  West,  Florida.  The 
testing  included  28  high  humidity  CFT  runs;  one  night,  three 
morning,  and  three  afternoon  high  humidity  Moving  Maritime 
Target  events,  and  48  CFT  runs. 

•  The  Navy  did  not  complete  any  free  flight  testing  in 
FY15  because  of  cancellations  due  to  weather  and  aircraft 
availability.  The  Navy  successfully  completed  one  free  flight 
test  event  on  October  2 1 ,  20 1 5 . 

Assessment 

•  The  Navy  began  operational  testing  of  the  JSOW  C-l  in  FY15. 
Preliminary  results  to  date  indicate: 

-  Weapon  impact  accuracy  for  moving  maritime  targets 
continues  to  be  well  within  the  accuracy  requirement  value 
and  accuracy  performance  against  stationary  land  targets 
has  been  maintained. 

-  JSOW  C-l  MFHBOMF  is  below  the  requirement  value. 
This  is  primarily  the  result  of  software-driven  problems. 
With  the  migration  from  F/A-18E/F  H8  OFP  to  FI  10, 
the  MFHBOMF  is  showing  progress  towards  meeting 
the  requirement  value  by  the  end  of  operational  testing. 
Achieving  adequate  assessment  of  MFHBOMF  during 
operational  testing  is  an  area  of  moderate  risk. 


-  The  Navy  has  reduced  the  complexity  of  the  PVI  in  the 
F/A-18E/F  H10  OFP.  There  remain  minor  PVI  challenges 
that  could  prevent  successful  mission  execution.  These 
challenges  can  be  effectively  overcome  with  proper 
training  prior  to  employment.  The  Navy  is  addressing 
these  challenges  in  F/A-18E/F  H12  OFP,  scheduled  for 
release  in  FY17. 

•  A  software  upgrade  for  the  Joint  Mission  Planning  System, 
version  3.0.2,  arrived  in  late  September.  This  is  expected 
to  resolve  issues  with  the  placement  of  the  network-enabled 
weapon  Link  16  network  load  for  the  JSOW  on  the  F-18,  but 
this  has  not  yet  been  verified. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  partially 
addressed  the  previous  recommendations.  The  Navy  has 
demonstrated  a  reduction  in  software-driven  failures  during 
the  extended  integrated  testing  phase.  While  they  have 
significantly  reduced  the  complex  PVI,  their  plan  will  not 
fully  address  this  issue  until  the  F/A-18E/F  H12  OFP  release, 
scheduled  for  FY17. 

•  FY15  Recommendation. 

1.  The  Navy  should  continue  to  reduce  the  PVI  complexity 
between  the  JSOW  C-l  and  the  F/A-18E/F  to  permit 
successful  mission  execution. 
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LHA  6  New  Amphibious  Assault  Ship  (formerly  LHA(R)) 


Executive  Summary 

•  The  Navy  identified  a  budget  shortfall  in  September  2014 
that  prohibits  the  completion  of  IOT&E  by  the  acquisition 
program’s  threshold  of  October  2016  for  reaching  Initial 
Operational  Capability.  The  late  delivery  of  the  ship  and  the 
3-month  extension  of  the  ship’s  Post  Shakedown  Availability 
(PSA)  reduced  its  availability  for  operational  testing  prior 

to  her  deployment  in  FY17.  The  Navy  and  Marine  Corps 
Operational  Test  Agencies  developed  a  plan  to  complete 
Amphibious  Warfare  (AMW)  IOT&E  in  conjunction  with 
scheduled,  pre-deployment  fleet  exercises.  The  Navy’s 
Program  Office  is  also  seeking  an  agreement  with  fleet 
and  Marine  Corps  leadership  to  conduct  the  Total  Ship 
Survivability  Trial  (TSST)  in  conjunction  with  these  fleet 
exercises  to  support  an  operationally  realistic  load  out  of  the 
ship  during  the  IOT&E. 

•  The  Navy  commenced  LHA  6  IOT&E  with  a  test  of  the  Air 
Warfare  Ship  Self-Defense  System  (SSDS).  LHA  6  combat 
system’s  testing  was  partially  accomplished  with  mixed 
results.  Challenges  to  combat  system  effectiveness  persist 
and  lead  to  questions  about  the  platform’s  ability  to  defend 
against  some  threats.  Several  events  remain  outstanding  for 
the  completion  of  the  approved  test  plans. 

•  LHA  6  entered  her  PSA  on  May  26,  2015,  with  an  anticipated 
March  25,  2016  completion  date.  The  Navy  will  implement 
changes  necessary  for  Joint  Strike  Fighter  (JSF)  that  will 
also  benefit  the  incorporation  of  MV-22  Osprey  operations 
on  the  LHA  6  during  her  PSA  and  will  include  these  changes 
into  the  LHA  7  construction  plan.  LHA  6  will  conduct  her 
maiden  deployment  in  mid-20 1 7  with  a  standard  Marine 
Expeditionary  Unit  Aviation  Combat  Element  (MEU  ACE) 
that  includes  AV-8B  Harrier  aircraft.  LHA  6  will  complete 
operational  testing  with  an  evaluation  of  the  ship’s  ability  to 
support  a  complement  of  20xJSF  aircraft  in  FY19. 

System 

•  LHA  6  is  a  large-deck  amphibious  assault  ship  designed 
to  support  a  notional  mix  of  fixed  and  rotary-wing 
aircraft  consisting  of  12  MV-22  Ospreys,  6  F-35B  JSFs 
(Short  Take -Off/Vertical  Landing  variant),  4  CH-53Es, 

7  AH-ls/UH-ls,  and  2  embarked  H-60  Search  and  Rescue 
aircraft,  or  a  load  out  of  20  F-35Bs  and  2  embarked  H-60 
Search  and  Rescue  aircraft.  Key  ship  features  and  systems 
include  the  following. 

-  Greater  aviation  storage  capacity  and  an  increase  in  the 
size  of  the  hangar  bay,  which  is  necessary  to  accommodate 
the  enhanced  aviation  maintenance  requirements  for 
the  MEU  ACE  with  F-35B  and  MV-22.  Additionally, 
two  maintenance  areas  with  high-overhead  clearance  are 
incorporated  into  the  design  of  the  ship  to  accommodate 


maintenance  on  MV-22s  in  the  spread  configuration  (wing 
spread,  nacelles  vertical  and  rotors  spread). 

-  Shipboard  medical  spaces  are  reduced  by  approximately 
two  thirds  compared  to  contemporary  LHDs  to  expand  the 
hangar  bay. 

-  The  combat  system  includes  the  SSDS  MK  2,  the  Phalanx 
Close-In  Weapon  System  Block  IB,  and  the  MK  38  Mod  2 
Gun  Weapon  System  for  defense  against  air  threats  and 
small  surface  craft.  The  SSDS  MK  2-based  combat  system 
integrates  with  the  following  five  major  components: 

-  The  SSDS  MK  2  Mod  4B  control  and  decision  system 
supports  the  integration  and  control  of  most  other  combat 
system  elements 

-  The  ship’s  AN/SPS-48E  and  AN/SPS-49A  air  search  radars 
and  the  AN/SPQ-9B  horizon  search  radar 

-  USG-2  Cooperative  Engagement  Capability  radar  tracking 
systems 

-  The  Rolling  Airframe  Missile  and  the  Evolved  Seasparrow 
Missile  (ESSM),  with  the  NATO  Seasparrow  MK  9  Track 
Illuminators 

-  The  AN/SLQ-32B(V)2  electronic  warfare  system  with  the 
Nulka  electronic  decoy-equipped  MK  53  Decoy  Launching 
System 

•  Two  marine  gas  turbine  engines,  two  electric  auxiliary 
propulsion  motors,  and  two  controllable  pitch  propellers 
provide  propulsion.  Six  diesel  generators  provide  electric 
power. 

•  Command,  Control,  Communications,  Computers,  and 
Intelligence  (C4I)  facilities  and  equipment  to  support  Marine 
Corps  Landing  Force  operations  are  part  of  the  program  of 
record. 
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•  LHA  8  is  the  lead  ship  for  the  Flight  1  variant  of  the  LHA(R) 
Amphibious  Assaidt  Ship  replacement  program.  It  has  a 
modified  flight  deck  and  reduced  island  intended  to  enable  an 
aviation  support  capability  similar  to  that  of  LHA  6.  LHA  8 
also  includes  a  well  deck  for  deploying  surface  connectors. 

Mission 

The  Joint  Maritime  Component  Commander  will  employ  LHA  6 
to: 

•  Serve  as  the  primary  aviation  platform  within  an  Amphibious 
Ready  Group  with  space  and  accommodations  for  Marine 
Corps  vehicles,  cargo,  ammunition,  and  more  than 

1,600  troops 


•  Serve  as  an  afloat  headquarters  for  a  MEU  Amphibious 
Squadron,  or  other  Joint  Force  commands  using  its  C4I 
facilities  and  equipment 

•  Accommodate  elements  of  a  Marine  Expeditionary  Brigade 
when  part  of  a  larger  amphibious  task  force 

•  Carry  and  discharge  combat  service  support  elements  and 
cargo  to  sustain  the  landing  force 

Major  Contractor 

Huntington  Ingalls  Industries,  Ingalls  Shipbuilding 

Division  -  Pascagoula,  Mississippi 


Activity 

•  DOT&E  observed  an  enhanced  acoustic  trial  on  the  LHA  6 
in  December  2014.  This  trial  collected  keel  aspect  acoustic 
signatures  in  addition  to  the  beam  aspect  signatures.  The  data 
will  be  used  by  the  Navy  for  the  ship’s  mine  susceptibility 
trial. 

•  DOT&E  observed  the  Navy’s  Combat  Systems  Ships 
Qualification  Trials  for  LHA  6  from  March  9  -  13,  2015, 
and  Final  Contractor  Trials  (FCTs)  from  March  30  through 
April  2,  2015.  These  events  are  part  of  the  developmental 
test  strategy  described  in  the  DOT&E-approved  Test  and 
Evaluation  Master  Plan  (TEMP). 

•  The  Navy  commenced  LHA  6  IOT&E  onboard  LHA  6 
with  Enterprise  Test  (ET)  06  of  the  Air  Warfare  Ship  Self- 
Defense  capability.  The  test  was  conducted  in  accordance 
with  a  DOT&E-approved  test  plan  from  April  20  through 
May  1,  2015,  and  produced  mixed  results. 

•  The  Navy  released  a  Request  for  Proposal  on  June  25,  2015 
directly  to  General  Dynamics  National  Steel  and  Shipbuilding 
Company  (commonly  referred  to  as  NASSCO)  and 
Huntington  Ingalls  Industries  for  the  construction  of  LHA  8. 
The  two  shipbuilders  are  deemed  by  the  Service  to  be  the  only 
two  in  the  country  capable  of  building  amphibious  assault 
ships.  The  Navy  anticipates  responses  for  the  LHA  8  Request 
for  Proposal  in  FY16. 

•  The  Navy  is  developing  an  LHA(R)  TEMP  Revision  B 
to  address  design  modifications  to  LHA  8  to  include  the 
addition  of  the  well  deck  and  changes  to  the  flight  deck,  the 
island  configuration,  the  combat  system,  medical  spaces, 
fuel  tanks,  and  supporting  spaces.  Marine  Corps  aircraft, 
surface  connectors,  and  vehicles  are  also  evolving.  LHA  8  is 
intended  to  bring  new  operational  capability  to  the  fleet,  which 
requires  an  AMW IOT &E  to  account  for  the  advancements  in 
modernization  from  LHA  6. 

•  In  May  2015,  the  Navy  delivered  a  draft  Vulnerability 
Assessment  Report  (VAR)  for  the  LHA  6,  and  is  using  the 
findings  in  its  planning  for  LHA  6’s  TSST.  The  Navy  has 
stated  it  is  not  planning  to  execute  the  Advanced  Mine 
Simulation  System  (AMISS)  trial,  which  would  be  used  to 
establish  the  mine  susceptibility  of  the  LHA  6,  as  agreed  to  in 


the  DOT&E-approved  TEMP  Revision  A.  To  date,  the  Navy 
has  not  presented  a  valid  alternative  to  conducting  the  AMISS 
trial. 

•  LHA  6  is  scheduled  to  complete  her  PSA  on  March  25,  2016. 
The  principle  work  accomplished  during  PSA  are  the  design 
modifications  to  the  flight  deck  to  account  for  the  deck 
strengthening,  heat-resistant  material  improvements,  and 
lighting  positioning  to  accommodate  the  JSF  F-35B  and 
benefit  MV-22  Osprey  operations.  The  flight  deck  changes 
have  been  included  in  the  LHA  7  design  currently  under 
construction  at  Huntington  Ingalls  shipyard. 

Assessment 

•  The  late  delivery  of  the  ship,  the  3 -month  extension  of  the 
ship’s  PSA,  and  the  additional  requirements  to  serve  as  the 
JSF  F-35B  test  platform,  further  reduce  LHA  6’s  availability 
for  dedicated  operational  testing  prior  to  her  deployment  in 
FY17. 

•  The  Navy  identified  a  budget  shortfall  in  September  2014 
that  will  prevent  the  completion  of  IOT&E  prior  to  the 
acquisition  program  threshold  of  October  2016.  The  Navy  has 
developed  an  alternate  path  to  complete  the  required  testing  in 
coordination  with  LHA  6’s  pre-deployment  fleet  exercises. 

•  During  Combat  Systems  Ships  Qualification  Trials  and  FCT 
events,  the  ship  performed  well  during  deck  (anchoring 
drop  test),  engineering  (main  propulsion  and  mobility),  and 
operational  evolutions  (flight  deck).  The  ship  completed  FCTs 
with  an  overall  assessment  of  satisfactory. 

•  Combat  system  testing  in  LHA  6  during  ET-06  and  on  the 
Self-Defense  Test  Ship  during  ET-05  (the  Self-Defense 
Test  Ship  was  set-up  in  the  LHA  6  configuration  for  ET-05) 
produced  mixed  results.  Several  challenges  persist  to  the 
efficacy  of  the  ship’s  combat  system  against  all  threats. 
Integration  and  combat  system  shortfalls  associated  with 
the  govermnent  furnished  equipment  provided  to  SSDS 
MK  2-based  combat  systems  (as  discussed  in  the  DOT&E 
Self-Defense  Operational  Mission  Capability  Report  dated 
November  5,  2012)  must  be  resolved  to  permit  LHA  6  to 
satisfy  its  Probability  of  Raid  Annihilation  (P  )  requirement. 
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This  is  the  greatest  risk  to  the  ship  successfully  demonstrating 
its  survivability. 

•  Because  LHA  6  does  not  have  a  well-deck,  it  will  rely 
exclusively  on  air  assets  to  move  forces  ashore.  The  Navy  and 
Marine  Corps  are  in  the  process  of  adjusting  their  tactics  to 
optimize  the  capabilities  of  LHA  6.  In  particular,  the  aircraft 
mix  and  equipment  load-out  used  on  an  LHD  may  not  be 
optimal  to  rapidly  mass  combat  power  ashore  from  LHA  6. 

The  Navy  and  Marine  Corps  need  to  finalize  their  tactics  prior 
to  the  phase  of  10T&E  in  which  they  will  be  used. 

•  LHA  6  TSST,  which  contributes  to  the  survivability 
assessment  of  the  ship,  was  planned  to  occur  during  the  AMW 
event  to  minimize  the  cost  of  the  test  program.  The  Navy  has 
rescheduled  the  test  to  occur  just  before  the  LHA  6  AMW 
Marine  Integrated  Training  exercise,  which  is  projected  to 
occur  in  March  2017.  The  Navy  is  coordinating  with  the  fleet 
and  Marine  Corps  leadership  to  ensure  the  TSST  is  conducted 
in  an  operationally  realistic  manner. 

•  The  LHA  6  SSDS  has  demonstrated  capability  against 
some  classes  of  Anti-Ship  Cruise  Missile  (ASCM)  threats. 
However,  based  on  combat  system’s  testing  on  LHA  6  and 
other  platforms,  it  is  unlikely  that  LHA  6’s  SSDS  MK  2-based 
combat  system  will  meet  the  ship’s  PR ,  requirement  against  all 
classes  of  ASCMs. 

-  The  Navy  initiated  the  Fire  Control  Loop  Improvement 
program  (FCLIP)  to  correct  some  combat  system 
deficiencies  related  to  self-defense  against  ASCMs  and  has 
the  potential  to  mitigate  some  of  the  vulnerabilities. 

-  However,  The  Navy  has  completed  Phase  1  of  the  FCLIP. 
What  was  formally  known  as  FCLIP  Phase  2  and  3  are  now 
merged  into  FCLIP  Phase  2,  which  is  not  yet  funded. 

•  The  MK  29-guided  missile  launch  system  used  to  launch  the 
ESSM  experienced  several  motor  failures  during  ET-06.  The 
ship’s  crew  had  to  replace  vertical  train  servo  motors  in  both 
of  the  MK  29  mounts.  Although  this  was  just  an  inconvenient 
delay  for  test  activities,  motor  failures  during  an  unexpected 
attack  could  degrade  the  combat  system’s  ability  to  intercept 
incoming  threats  with  the  ESSM.  The  Navy  attributes  the 
problem  to  the  increased  weight  of  the  ESSM  relative  to  the 
seasparrow  missile  it  replaces.  The  Navy  is  investigating 
modifying  or  replacing  the  MK  29  launch  system  to  handle 


the  extra  weight.  In  the  short  term,  the  Navy  is  addressing  the 
problem  through  procedural  (e.g.,  the  use  of  counterweights 
when  loading  an  ESSM  into  the  launcher)  and  logistical  (e.g., 
supplying  crews  with  spare  motors)  efforts. 

•  In  July  2015,  the  Navy  identified  it  was  having  problems 
developing  the  needed  Multi-Stage  Supersonic  Target.  After 
a  review  of  the  options  to  support  the  planned  IOT&E  event, 
DOT&E  determined  the  Navy  did  not  have  an  affordable  or 
viable  target  development  plan  and  its  alternative  solution 
was  not  adequate.  Hence,  DOT&E  recommended  the  Navy 
cease  the  Multi-Stage  Supersonic  Target  development  until  the 
requirements  are  better  defined  and  an  affordable  solution  can 
be  found. 

•  DOT&E  does  not  agree  that  the  Navy’s  proposed  modeling 
and  simulation-based  approach  to  assessing  the  mine 
susceptibility  of  LHA  6  is  adequate.  The  Navy  should 
therefore  plan  to  execute  the  AMISS  trial  as  agreed  to  in  the 
DOT&E-approved  TEMP  Revision  A. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  either 
addressed  or  established  a  process  through  which  to  address 
most  of  the  previous  recommendations.  However,  the  Navy 
has  not  fully  resolved  the  recommendation  to  correct  systems 
engineering  deficiencies  related  to  SSDS  MK  2-based  combat 
systems  and  other  combat  system  deficiencies  so  that  LHA  6 
can  satisfy  its  PRA  requirement. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Allocate  sufficient  resources  to  permit  the  IOT&E  and 
TSST  to  be  conducted  as  a  stand-alone  event  for  LHA  8 
until  the  plan  for  executing  AMW  IOT&E  and  the  TSST 
in  coordination  with  fleet-exercises  can  be  evaluated  for 
LHA  6. 

2.  Resolve  the  MK  29  launcher  system  motor  failures  due  to 
the  additional  weight  of  the  ESSM. 

3.  The  Navy  and  Marine  Corps  need  to  finalize  their  tactics, 
techniques,  and  procedures  for  LHA  6  prior  to  the  phase  of 
IOT&E  in  which  they  will  be  used. 

4.  The  Navy  should  plan  and  resource  the  mine  susceptibility 
trial  for  the  LHA  6  using  the  AMISS. 
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Light  Armored  Vehicle  -  Anti-Tank  Modernization 

(LAV-ATM) 


Executive  Summary 

•  From  January  12  through  March  4,  2015,  the  Marine  Corps 
Operational  Test  &  Evaluation  Activity  (MCOTEA)  conducted 
an  operational  assessment  (O  A)  for  the  Light  Armored 
Vehicle  Anti-Tank  Modernization  (LAV-ATM)  using  two 
pre-Milestone  C  LAV-ATM  prototypes  in  accordance  with 

a  DOT&E-approved  test  plan.  Operational  Marines  from 
1st  Marine  Division  participated  in  the  OA. 

•  The  LAV-ATM  demonstrated  a  turret  reliability  of  0.95, 
exceeding  the  requirement  of  0.9  and  an  operational 
availability  of  0.965  exceeding  the  requirement  of  0.85. 
Because  of  the  short  duration  of  the  OA,  it  cannot  be 
concluded  with  statistical  confidence  that  the  reliability 
requirement  has  been  met. 

•  The  LAV-ATM  met  its  classified  probability  of  hit 
requirements. 

System 

•  The  LAV  Family  of  Vehicles  shares  a  common-base  platform 
configuration  and  consists  of  seven  variants. 

•  The  LAV-ATM  variant  fires  tube-launched,  optically-tracked, 
wire-guided  anti-armor  missiles. 

•  The  eight-wheeled  LAVs  have  armor  to  protect  units  from 
some  small  arms,  light  machine  gun  fire,  and  artillery 
projectile  fragments. 

•  LAVs  are  capable  of  operating  on  primary  roads,  trails,  and 
cross-country  terrain. 

•  LAVs  have  a  limited  swim  capability  for  maneuvering 
waterways  and  river  crossings. 

Mission 

Marine  Corps  commanders  will  use  Light  Armored 

Reconnaissance  (LAR)  battalions  equipped  with  LAV-ATM 


vehicles  to  conduct  reconnaissance,  security,  economy  of  force 

operations,  and  limited  offensive  or  defensive  operations. 

•  During  offensive  operations,  the  Marine  Corps  will  employ 
the  LAV-ATM  to  provide  anti-armor  fires  that  support 
maneuvering  LAR  companies  and  platoons. 

•  During  defensive  operations,  the  Marine  Corps  will  use  the 
LAV-ATM  to  provide  LAR  companies  and  platoons  with 
long-range,  stand-off  anti-armor  fires,  and  observation  in  all 
climate  conditions  and  periods  of  limited  visibility. 

Major  Contractor 

Raytheon  -  McKinney,  Texas 


Activity 

•  From  January  12  through  March  4,  2015,  MCOTEA 
conducted  an  OA  for  the  LAV-ATM  at  the  Marine  Corps 
Air  Ground  Combat  Center,  Twentynine  Palms,  California, 
using  two  pre-Milestone  C  LAV-ATM  prototypes. 
Operational  Marines  from  1st  Marine  Division  participated 
in  the  OA.  Testing  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan. 

•  The  OA  consisted  of  a  training  phase,  a  weapon  systems 
comparison  test,  and  a  field  firing  test. 

-  The  training  phase  included  a  40-hour  new  equipment 
training  period  for  operators  and  maintainers. 


-  The  weapon  systems  comparison  test  phase  included 
four  48-hour  operational  mission  profiles.  Mission  routes 
covered  desert  terrain  that  included  ravines,  washouts, 
hills,  rocks,  and  soft  sand.  In  each  operational  mission 
profile,  each  platoon  conducted  eight  missions,  divided 
between  offense  and  defense  and  between  day  and  night. 

-  The  field  firing  test  phase  of  LAV-ATM  consisted  of  44  live 
tube-launched,  optically-tracked,  wire-guided  missile 
firings  on  an  instrumented  range  and  tactical  movements 
conducted  between  missile  firings  to  create  operational 
stresses  on  the  crews  and  system. 
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Assessment 

•  During  the  OA,  the  LAV- ATM’s  second  generation, 
forward-looking  infrared  thermal  sight  demonstrated  the 
following  capabilities: 

-  Enabled  gunners  to  acquire  targets  at  greater  ranges 

-  Provided  an  eye-safe  laser  range  finder 

-  Provided  automatic  boresighting 

-  Aided  the  gunner  in  target  tracking 

-  Incorporated  an  embedded  training  capability 

•  The  LAV- ATM  can  maneuver  with  its  turret  raised,  providing 
a  capability  to  detect  and  acquire  targets  on  the  move  and 
engage  targets  more  quickly  than  the  current  LAV- AT  that 
takes  two  minutes  to  raise  its  turret.  However,  due  to  test 
limitations  including  the  lack  of  the  capability  to  conduct 
real-time  casualty  assessment,  the  OA  did  not  provide 
results  actually  demonstrating  the  operational  benefit  of  this 
improvement. 

•  The  new  vehicle  commander’s  sight  provides  the  vehicle 
commander  with  the  same  imagery  seen  by  the  gunner 
to  improve  target  detection,  acquisition,  recognition,  and 
identification  and  to  increase  the  vehicle  commander’s 


situational  awareness.  Again,  test  limitations  precluded 
demonstrating  the  actual  operational  benefit  of  this 
improvement. 

•  The  LAV-ATM  demonstrated  a  turret  reliability  of  0.95, 
exceeding  the  requirement  of  0.9  and  an  operational 
availability  of  0.965,  exceeding  the  requirement  of  0.85. 
Because  of  the  short  duration  of  the  OA  it  cannot  be  concluded 
with  statistical  confidence  that  the  reliability  requirement  has 
been  met. 

•  The  LAV-ATM  met  its  classified  probability  of  hit 
requirements. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY15  Recommendation. 

1 .  The  Marine  Corps  should  field  the  instrumentation 
capabilities,  including  real-time  casualty  assessment, 
necessary  to  conduct  useful  realistic  operational  testing  of 
new  combat  vehicles  and  upgrades  to  existing  vehicles. 
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Littoral  Combat  Ship  (LCS)  and 
Associated  Mission  Modules 


Executive  Summary 

•  In  the  report  to  Congress  required  by  the  National  Defense 
Authorization  Act  (NDAA)  for  FY15,  DOT&E  concluded 
that  the  now-planned  use  of  the  Littoral  Combat  Ship  (LCS) 
as  a  forward-deployed  combatant,  where  it  might  be  involved 
in  intense  naval  conflict,  appears  to  be  inconsistent  with  its 
inherent  survivability  in  those  same  environments. 

•  This  same  report  also  concluded  that  the  ability  of  LCS  to 
successfully  execute  significant  aspects  of  its  envisioned 
concept  of  operations  (CONOPS)  depends  on  the  effectiveness 
of  the  mission  packages.  To  date,  the  Navy  has  not  yet 
demonstrated  effective  capability  for  either  the  Mine 
Countermeasures  (MCM)  or  Anti-Submarine  Warfare  (ASW) 
mission  packages.  The  Surface  Warfare  (SUW)  mission 
package  has  demonstrated  a  modest  ability  to  aid  the  ship  in 
defending  itself  against  small  swarms  of  small  boats,  and  the 
ability  to  conduct  maritime  security  operations. 

•  During  FY15,  the  Navy  conducted  developmental  testing 
of  the  Independence  variant  LCS  seaframe  and  Increment  1 
MCM  mission  package  aboard  USS  Independence  (LCS  2). 
Although  the  Navy  intended  to  complete  that  testing  by 
June  2015  and  conduct  the  operational  test  from  July  to 
September,  it  extended  developmental  testing  through  the  end 
of  August  because  of  seaframe  failures  and  MCM  mission 
system  reliability  shortfalls.  The  Navy  subsequently  decided 
in  October  2015  to  postpone  the  first  phase  of  10T&E  of  the 
MCM  mission  package  until  sometime  in  2016,  at  the  earliest. 

•  The  Navy  chartered  an  independent  program  review  of  the 
Remote  Minehunting  System  (RMS),  including  an  evaluation 
of  potential  alternative  MCM  systems,  in  September  2015. 

•  DOT&E  concluded  in  a  November  2015  memorandum  to  the 
USD(AT&L)  and  the  Navy,  based  on  all  testing  conducted 

to  date,  that  an  LCS  employing  the  current  MCM  mission 
package  would  not  be  operationally  effective  or  operationally 
suitable  if  the  Navy  called  upon  it  to  conduct  MCM 
missions  in  combat  and  that  a  single  LCS  equipped  with  the 
Increment  1  MCM  mission  package  would  provide  little  or  no 
operational  capability  to  complete  MCM  clearance  missions  to 
the  levels  needed  by  operational  commanders.  The  following 
summarize  the  primary  reasons  for  this  conclusion: 

-  Critical  MCM  systems  are  not  reliable. 

-  The  ship  is  not  reliable. 

-  Vulnerabilities  of  the  Remote  Multi-Mission  Vehicle 
(RMMV)  to  mines  and  its  high  rate  of  failures  do  not 
support  sustained  operations  in  potentially  mined  waters. 

-  RMMV  operational  communications  ranges  are  limited. 

-  Minehunting  capabilities  are  limited  in  other-than-benign 
environmental  conditions. 

-  The  fleet  is  not  equipped  to  maintain  the  ship  or  the  MCM 
systems. 


Freedom  Variant  (LCS  1) 


Independence  Variant  (LCS  2) 


-  The  Airborne  Mine  Neutralization  Systems  (AMNS) 
cannot  neutralize  most  of  the  mines  in  the  Navy’s  threat 
scenarios;  an  Explosive  Ordinance  Disposal  Team  or  other 
means  provided  by  another  unit  must  be  used. 

•  During  the  MCM  mission  package  Technical  Evaluation 
(TECHEVAL),  the  Navy  demonstrated  that  an  LSC  could 
detect,  classify,  identify,  and  neutralize  only  a  fraction  of  the 
mines  in  the  Navy’s  mine  clearance  scenarios  while  requiring 
extraordinary  efforts  from  shore  support,  maintenance 
personnel,  and  contractors. 

•  The  Navy  also  conducted  both  developmental  and  operational 
testing  of  the  Independence  variant  LCS  seaframe  with 

the  Increment  2  SUW  mission  package  aboard  LCS  4. 
Operational  testing  of  the  seaframe  and  Increment  2  SUW 
mission  package  is  not  yet  complete  because  of  pending 
changes  to  the  ship’s  air  defense  system,  Sea  Rolling  Airframe 
Missile  (SeaRAM),  and  other  elements  of  the  ship’s  combat 
system  and  networks.  A  second  phase  of  operational  testing 
of  the  Increment  2  version  of  the  SUW  mission  package 
and  Independence  variant  seaframe  is  scheduled  to  occur  in 
3QFY16. 
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•  While  equipped  with  the  Increment  2  SUW  mission  package, 
LCS  4  participated  in  three  engagements  with  small  swarms 
of  Fast  Inshore  Attack  Craft  (FIAC).  Although  all  of  the 
attacking  boats  were  ultimately  defeated,  an  attacker  managed 
to  penetrate  the  “keep-out”  range  in  two  of  the  three  events.  In 
all  three  events,  however,  the  ship  expended  a  large  quantity 
of  ammunition  from  the  seaframe’s  57  mm  gun  and  the 

two  mission  package  30  mm  guns,  while  contending  with 
repeated  network  communication  faults  that  disrupted  the 
flow  of  navigation  information  to  the  gun  systems  as  well 
as  azimuth  elevation  inhibits  that  disrupted  or  prevented 
establishing  firing  solutions  on  the  targets.  LCS  4’s  inability 
to  defeat  this  relatively  modest  threat  beyond  the  “keep-out” 
range  routinely  under  test  conditions  raises  questions  about 
its  ability  to  deal  with  more  challenging  threats  that  could  be 
present  in  an  operational  environment. 

•  In  comparison  to  other  Navy  ships,  the  LCS  seaframes  have 
relatively  modest  air  defense  capabilities  that  cannot  be 
characterized  fully  until  planned  tests  on  LCS  7  and  LCS  8  and 
the  Navy’s  unmanned  self-defense  test  ship  provide  data  for 
the  Navy  Probability  of  Raid  Annihilation  (P  )  high-fidelity 
modeling  and  simulation  analyses.  The  Navy  plans  to  begin 
those  tests  in  FY17.  In  FY15,  DOT&E  learned  that  the 
Program  Executive  Office  for  Integrated  Warfare  Systems 
(PEO  IWS)  stopped  work  on  the  PRA  Test  Bed  for  the  Freedom 
variant  because  a  high-fidelity  model  of  the  ship’s  AN/SPS-75 
radar  was  not  being  developed.  Development  of  an  acceptable 
radar  model  requires  intellectual  property  rights  that  the 
Navy  does  not  hold  and  is  not  actively  seeking.  Although 
less  critical  because  of  the  combat  system  architecture  of 

the  Independence  variant,  the  Navy  has  also  been  unable 
to  develop  a  high-fidelity  model  of  that  ship’s  AN/SPS-77 
radar  for  the  same  reason.  In  an  August  2015  memorandum, 
DOT&E  advised  Navy  officials  that  the  lack  of  these  radar 
models  threatens  the  viability  of  the  Navy’s  strategy  for 
evaluation  of  LCS  air  defense  capabilities  and  suggested 
alternative  strategies  specific  to  each  seaframe  variant.  The 
Navy  has  not  decided  what  course  of  action  it  wants  to  pursue. 

•  In  August  2015,  the  Navy  conducted  the  first  shipboard  live 
firing  of  the  ship’s  SeaRAM  system.  The  demonstration  was 
not  designed  to  be  an  operationally  realistic  test  of  the  ship’s 
capability.  The  aerial  drone’s  flight  profile  and  configuration 
were  not  threat  representative. 

•  Test  activities  in  FY 15  allowed  the  collection  of  reliability, 
maintainability,  availability,  and  logistics  supportability  data 
to  support  evaluation  of  the  operational  suitability  of  the 
Independence  variant  seaframe.  Although  incomplete,  the 
data  collected  to  date  show  that  many  of  the  Independence 
variant  seaframe  systems  have  significant  reliability  problems. 
During  developmental  testing,  the  LCS  4  crew  had  difficulty 
keeping  the  ship  operational  as  it  suffered  repeated  failures  of 
the  ship’s  diesel  generators,  water  jets,  and  air  conditioning 
units.  LCS  4  spent  45  days  over  a  period  of  1 13  days  without 
all  4  engines  and  steerable  water  jets  operational.  This 
includes  a  19-day  period  in  May  when  3  of  the  4  engines 
were  degraded  or  non-functional.  During  the  five-month 


MCM  mission  package  TECHEVAL  period,  LCS  2  seaframe 
failures  caused  the  ship  to  return  to,  or  remain  in,  port  for 
repairs  on  seven  occasions.  Similar  to  LCS  4,  the  ship’s  core 
systems,  such  as  the  air  defense  system,  SeaRAM,  the  MK  110 

57  mm  gun,  the  electro-optical/infrared  sensor  (Sea  Star 
Shipboard  Airborne  Forward-Looking  Infra-Red  Equipment 
(SAFIRE))  used  to  target  the  gun,  and  the  ship’s  primary  radar, 
experienced  failures,  leaving  the  ship  with  no  air  or  surface 
defense  capability  for  more  than  one-half  of  the  test  period. 
LCS  2  was  unable  to  launch  and  recover  RMMVs  on  15  of  the 

58  days  underway  because  of  4  separate  propulsion  equipment 
failures  involving  diesel  engines,  water  jets,  and  associated 
hydraulic  systems  and  piping. 

•  The  Navy  conducted  the  first  of  four  periods  of  cybersecurity 
testing  on  the  Independence  variant  while  the  ship  was 
moored  in  Pensacola,  Florida,  during  a  comprehensive 
maintenance  availability.  The  test  comprised  a  Cooperative 
Vulnerability  and  Penetration  Assessment  (CVPA)  of  the 
seaframe  and  embarked  Increment  1  MCM  mission  package. 
The  CVPA  details  are  classified  but  indicate  that,  like  the 
Freedom  variant  seaframe,  the  Independence  variant  seaframe 
has  cybersecurity  deficiencies  that  significantly  degrade 
operational  effectiveness.  Plans  for  the  remaining  period 

of  the  cybersecurity  testing  in  LCS  2  are  on  hold  pending 
a  Navy  decision  on  the  readiness  of  the  Increment  1  MCM 
mission  package  and  Independence  variant  seaframe  for 
MCM  operational  testing.  The  Navy  delayed  the  two  periods 
of  cybersecurity  testing  in  LCS  4  until  after  it  completes 
an  upgrade  of  the  ship’s  networks  designed  to  enhance 
cybersecurity  and  correct  known  issues. 

•  DOT&E  does  not  expect  either  LCS  variant  to  be  survivable 
in  high-intensity  combat  because  the  design  requirements 
accept  the  risk  that  the  crew  would  have  to  abandon  ship 
under  circumstances  that  would  not  require  such  action  on 
other  surface  combatants.  Although  the  ships  incorporate 
capabilities  to  reduce  their  susceptibility  to  attack,  previous 
testing  of  analogous  capabilities  demonstrates  it  cannot  be 
assumed  LCS  will  not  be  hit  in  high-intensity  combat. 

•  The  LCS  3  Total  Ship  Survivability  Trial  (TSST)  revealed 
significant  deficiencies  in  the  Freedom  variant  design.  Much 
of  the  ship’s  mission  capability  would  have  been  lost  because 
of  damage  caused  by  the  initial  weapons  effects  or  the  ensuing 
fire.  The  weapons  effects  and  fire  damage  happened  before 
the  crew  could  respond,  and  the  ship  does  not  have  sufficient 
redundancy  to  recover  the  lost  capability. 

System 

Seaframes 

•  The  LCS  is  designed  to  operate  in  the  shallow  waters  of 
the  littorals  that  can  constrain  the  ability  of  larger  ships  to 
maneuver. 

•  The  Navy  originally  planned  to  acquire  55  LCSs,  but 
reduced  the  planned  procurement  to  52  ships  in  2013.  In  a 
February  24,  2014  memorandum,  the  Secretary  of  Defense 
announced  that  no  new  contract  negotiations  beyond 

32  ships  would  go  forward  and  directed  the  Navy  to  submit 
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alternative  proposals  to  procure  a  more  capable  and  lethal 
small  surface  combatant,  generally  consistent  with  the 
capabilities  of  a  Frigate.  Further  discussion  of  the  small 
surface  combatant  variant  (now  called  a  Frigate)  is  in  a 
separate  article  in  this  annual  report. 

•  The  Navy  is  currently  procuring  two  variants  of  LCS 
seaframes: 

-  The  Freedom  variant  (odd-numbered  ships)  is  a 
semi-planing  monohull  design  constructed  of  steel 
(hull)  and  aluminum  (deckhouse)  with  two  steerable  and 
two  fixed-boost  water  jets  driven  by  a  combined  diesel 
and  gas  turbine  main  propulsion  system. 

-  The  Independence  variant  (even-numbered  ships)  is  an 
aluminum  trimaran  design  with  two  steerable  water  jets 
driven  by  diesel  engines  and  two  steerable  water  jets 
driven  by  gas  turbine  engines. 

•  Common  design  specifications  include: 

-  Sprint  speed  in  excess  of  40  knots,  draft  of  less 
than  20  feet,  and  an  un-refueled  range  in  excess  of 
3,500  nautical  miles  at  14  knots 

-  Accommodations  for  up  to  98  personnel 

-  A  common  Mission  Package  Computing  Environment 
(MPCE)  for  mission  package  control  using  Mission 
Package  Application  Software  (MPAS)  installed  when  a 
mission  package  is  embarked 

-  A  Multi- Vehicle  Communications  System  to  support 
simultaneous  communications  with  multiple  unmanned 
off-board  vehicles 

-  Flangars  sized  to  embark  MH-60R/S  and  Vertical 
Take-Off  Unmanned  Aerial  Vehicles  (VTUAVs) 

-  MK  110  57  mm  gun  (BAE/BOFORS) 

•  The  designs  have  different  core  combat  systems  to 
provide  command  and  control,  situational  awareness,  and 
self-defense  against  anti-ship  cruise  missiles  (ASCMs)  and 
surface  craft. 

-  Freedom  variant:  COMBATSS-21,  an  Aegis-based 
integrated  combat  weapons  system  with  a  TRS-3D 
(AN/SPS-75)  air  and  surface  search  radar  (ASR) 

(Airbus,  France),  Rolling  Airframe  Missile  (RAM) 
system  supported  by  elements  from  the  Ship 
Self-Defense  System  (Raytheon)  (one  21 -cell  launcher), 
a  Terma  Soft  Kill  Weapon  System  (Denmark),  and 

a  DORNA  EOD  gunfire  control  system  with  an 
electro-optical/infrared  sensor  (Navantia,  Spain)  to 
control  the  MK  110  57  mm  gun. 

-  Independence  variant:  Integrated  Combat  Management 
System  (derived  from  the  Thales  TACTICOS  system 
(The  Netherlands)  with  a  Sea  Giraffe  (AN/SPS-77)  ASR 
(SAAB,  Sweden),  one  MK  15  Mod  31  SeaRAM  system 
(Raytheon)  (integrates  the  search,  track,  and  engagement 
scheduler  of  the  Phalanx  Close-in  Weapon  System 
with  an  1 1-round  RAM  launcher  assembly),  ALEX 
(Automatic  Launch  of  Expendables)  System  (off-board 
decoy  countermeasures)  (Sippican,  U.S.),  and  SAFIRE 
(FLIR,  U.S.)  for  57  mm  gun  fire  control. 


Mission  Packages 

•  LCS  is  designed  to  host  a  variety  of  individual  warfare 
systems  (mission  modules)  assembled  and  integrated  into 
interchangeable  mission  packages.  The  Navy  currently 
plans  to  field  MCM,  SUW,  and  ASW  mission  packages.  A 
mission  package  provides  the  seaframes  with  capability 
for  a  single  or  “focused”  mission.  Multiple  individual 
programs  of  record  involving  sensor  and  weapon  systems 
and  off-board  vehicles  make  up  the  individual  mission 
modules.  Summarized  below  is  the  current  acquisition 
strategy  for  the  incremental  development  of  each  mission 
module.  However,  the  Navy  recently  began  an  effort  to 
revise  its  plan,  including  the  possibility  of  developing 
different  components  rather  than  some  upgrades. 

SUW  Mission  Package 

•  Increment  1  includes: 

-  Gun  Mission  Module  (two  MK  46  30  mm  guns) 

-  Aviation  Module  (embarked  MH-60R) 

•  Increment  2  adds: 

-  Maritime  Security  Module  (small  boats) 

•  Increment  3  is  expected  to  add: 

-  Surface-to-Surface  Missile  Module  Increment  I, 
employing  the  AGM  1 14L  Longbow  Hellfire  missile 

-  One  MQ-8C  Fire  Scout  VTUAV  to  augment  the  Aviation 
Module 

•  Increment  4,  if  fielded,  will  add: 

-  Surface-to-Surface  Missile  Module  Increment  II 
(replacing  Increment  I)  to  provide  a  longer  range  surface 
engagement  capability 

MCM  Mission  Package 

•  Increment  1  includes: 

-  Remote  Minehunting  Module,  consisting  of  two 
RMMVs  (version  6.0  (v6.0))  and  three  AN/AQS-20A 
sensors.  The  Navy  plans  to  incorporate  an  improved 
sensor  (AN/AQS-20C)  in  a  future  increment. 

-  Near  Surface  Detection  Module,  consisting  of 

two  Airborne  Laser  Mine  Detection  Systems  (ALMDS). 
The  Navy  plans  to  incorporate  improvements  in  a  future 
increment. 

-  Airborne  Mine  Neutralization  Module,  consisting  of 
two  AMNS  units.  In  Increment  1,  the  AMNS  does  not 
include  a  near  surface  mine  neutralization  capability. 

-  Aviation  Module  consisting  of  an  MH-60S  Block  2B  or 
subsequent  Airborne  Mine  Countermeasures  (AMCM) 
Helicopter  outfitted  with  an  AMCM  system  operator 
workstation  and  a  tether  system. 

•  Increment  2  is  expected  to  add: 

-  Coastal  Mine  Reconnaissance  Module,  consisting  of 
the  Coastal  Battlefield  Reconnaissance  and  Analysis 
(COBRA)  Block  I  system  and  one  MQ-8B  VTUAV 
for  daytime  unmanned  aerial  tactical  reconnaissance  to 
detect  and  localize  mine  lines  and  obstacles  in  the  beach 
zone. 
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•  Increment  3  is  expected  to  add: 

-  Unmanned  Mine  Sweeping  Module,  consisting  of 
the  Unmanned  Influence  Sweep  System  (UISS)  to 
actuate/ detonate  acoustic-,  magnetic-,  and  combined 
acoustic/magnetic-initiated  volume  and  bottom  mines  in 
shallow  water. 

-  Airborne  Mine  Neutralization  (Near- Surface)  Module 

•  Increment  4  is  expected  to  add: 

-  COBRA  Block  II  system,  which  retains  Block  I 
capability  and  adds  nighttime  minefield  and  obstacle 
detection  capability  and  day/night  detection  capability  in 
the  surf  zone. 

-  Buried  Minehunting  Module,  consisting  of  the  Knifefish 
Unmanned  Undersea  Vehicle,  a  battery-powered, 
autonomous  underwater  vehicle,  employing  a 
low-frequency,  broadband,  synthetic  aperture  sonar  to 
detect,  classify,  and  identify  volume  and  bottom  mines  in 
shallow  water. 

ASW  Mission  Package  (only  Increment  2) 

•  Torpedo  Defense  and  Countermeasures  Module 
(Lightweight  Tow  torpedo  countermeasure) 

•  ASW  Escort  Module  (Multi-Function  Towed  Array  and 
Variable  Depth  Sonar) 

•  Aviation  Module  (embarked  MH-60R  and  MQ-8B  Fire 
Scout  VTUAV)  (inclusion  of  Fire  Scout  is  reportedly  being 
deferred  because  of  fiscal  constraints.) 

Mission 

•  The  Maritime  Component  Commander  will  employ  LCS  to 
conduct  MCM,  ASW,  or  SUW  tasks  depending  on  the  mission 
package  installed  in  the  seaframe.  Because  of  capabilities 
inherent  to  the  seaframe,  commanders  can  employ  LCS  in 
a  maritime  presence  role  in  any  configuration.  With  the 
Maritime  Security  Module,  installed  as  part  of  the  SUW 


mission  package,  the  ship  can  conduct  Maritime  Security 
Operations,  including  Visit,  Board,  Search,  and  Seizure  of 
ships  suspected  of  transporting  contraband. 

•  The  Navy  can  employ  LCS  alone  or  in  company  with  other 
ships.  The  Navy’s  CONOPS  for  LCS  anticipates  that  the 
ship’s  primary  operational  role  will  involve  preparing  the 
operational  environment  for  joint  force  assured  access  to 
critical  littoral  regions  by  conducting  MCM,  ASW,  and 
SUW  operations,  possibly  under  an  air  defense  umbrella 
as  determined  necessary  by  the  operational  commander. 
However,  the  latest  CONOPS  observes,  “The  most  effective 
near-term  operational  roles  for  LCS  to  support  the  maritime 
strategy  are  theater  security  cooperation  and  MSO  [Maritime 
Security  Operations]  supporting  deterrence  and  maritime 
security.” 

Major  Contractors 

•  Freedom  variant  (LCS  1,  3,  5,  7,  and  follow-on  odd-numbered 
ships) 

-  Prime:  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Washington,  District  of  Columbia 

-  Shipbuilder:  Marinette  Marine  -  Marinette,  Wisconsin 

•  Independence  variant  (LCS  2,  4,  6,  8,  and  follow-on 
even-numbered  ships) 

-  Prime  for  LCS  2  and  LCS  4:  General  Dynamics 
Corporation  Marine  Systems,  Bath  Iron  Works  -  Bath, 
Maine 

-  Prime  for  LCS  6  and  follow-on  even  numbered  ships: 
Austal  USA  -  Mobile,  Alabama 

-  Shipbuilder:  Austal  USA  -  Mobile,  Alabama 

•  Mission  Packages 

-  Mission  Package  Integration  contract  awarded  to  Northrop 
Grumman  -  Los  Angeles,  California 


Activity 

LCS  Program 

•  In  February  2014,  the  Secretary  of  Defense  curtailed 
the  planned  Flight  0+  LCS  procurement  at  32  ships  and 
required  the  Navy  to  submit  alternative  proposals  for  a 
capable  small  surface  combatant  that  is  more  lethal  and 
survivable  than  the  current  LCS  design.  In  December  2014, 
the  Secretary  of  Defense  approved  the  Navy’s  proposal  to 
procure  a  small  surface  combatant  based  on  an  upgraded 
Flight  0+  LCS  with  minor  modifications. 

•  In  January  2015,  the  Secretary  of  the  Navy  announced 
that  the  modified  small  surface  combatant  LCS  would 
be  designated  a  Frigate  and  noted  that  the  Navy  would 
consider  re-designating  earlier  LCS  variants  as  Frigates 
if/when  they  receive  similar  modifications.  The  Navy 
began  work  on  a  Capabilities  Development  Document 
in  2015,  and  plans  to  complete  Joint  Staffing  of  the 
requirements  document  in  FY16.  Additional  information 


about  the  small  surface  combatant  (now  called  a  Frigate) 
modification  to  the  LCS  is  provided  in  a  separate  article  in 
this  annual  report. 

•  In  February  2015,  DOT&E  provided  the  Secretary  of  the 
Navy  certification  that  only  one  of  each  mission  module  is 
needed  to  support  operational  testing  in  compliance  with 
Section  122  of  the  NDAA  for  FY15. 

•  In  February  2015,  DOT&E  responded  to  the  reporting 
requirement  in  Section  124  of  the  FY15  NDAA,  which 
directed  DOT&E  to  report  on  the  Test  and  Evaluation 
Master  Plan  (TEMP)  for  LCS  seaframes  and  mission 
modules. 

•  In  April  2015,  DOT&E  provided  USD(AT&L)  an 
assessment  of  the  capabilities  and  limitations  of  LCS  ships 
and  mission  packages  to  support  USD(AT&L)’s  FY15  LCS 
Deep  Dive  and  annual  review  of  the  program.  That  report 
summarized  DOT&E ’s  current  assessment  of  both  variants, 
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including  an  evaluation  of  the  seaffames’  cybersecurity,  air 
defense,  surface  self-defense,  reliability,  and  availability, 
and  known  survivability  shortfalls.  The  report  also 
summarized  the  most  significant  concerns  for  each  of  the 
mission  packages  in  advance  of  the  planned  operational 
testing  of  both  the  SUW  and  MCM  mission  packages 
intended  to  occur  in  FY15. 

•  Also  in  April  2015,  DOT&E  submitted  a  report  to  Congress 
and  the  Secretary  of  Defense  responding  to  Section  123  of 
the  FY15  NDAA,  which  directed  DOT&E  and  the  Navy 

to  address  the  current  CONOPS  and  expected  survivability 
attributes  of  each  of  the  seaframes.  This  report  included  a 
review  of  the  survivability  testing,  modeling,  and  simulation 
conducted  to  date  on  the  two  seaframes,  and  an  assessment 
of  the  expected  survivability  of  LCS  in  the  context  of  its 
planned  employment  as  described  in  the  CONOPS. 

•  The  Navy  began  efforts  to  revise  the  LCS  TEMP  in 
4QFY15.  The  current  version  of  the  TEMP  was  only 
approved  for  the  testing  on  the  first  increment  of  the 
MCM  mission  package,  the  second  increment  of  the  SUW 
mission  package,  and  the  initial  ASW  mission  package. 

An  update  is  now  required  since  testing  of  the  Increment 

3  SUW  mission  package  is  expected  to  occur  in  FY16. 
Uncertainty  in  the  Navy’s  plans  for  the  mission  packages  as 
well  as  the  uncertainty  in  ship  availability  in  the  out  years 
is  slowing  the  TEMP’s  development.  The  FY16  NDAA 
directed  the  Navy  to  submit  a  current  TEMP  for  the  LCS 
mission  modules,  approved  by  DOT&E,  which  includes 
the  performance  levels  expected  to  be  demonstrated  during 
developmental  testing  for  each  component  and  mission 
module  prior  to  commencing  the  associated  operational  test 
phase. 

•  In  August  2015,  DOT&E  advised  Navy  officials  of 
concerns  that  the  Navy’s  current  lack  of  access  to  the 
intellectual  property  needed  to  develop  high-fidelity 
models  of  the  AN/SPS-75  and  AN/SPS-77  radars  for  use 
in  the  PRA  modeling  and  simulation  test  bed  will  preclude 
adequate  evaluation  of  LCS  air  defense  capabilities.  The 
memorandum  detailed  alternative  test  strategies  involving 
additional  live  testing  that  might  be  acceptable  should  the 
Navy  be  unable  to  obtain  the  necessary  data  rights. 

•  In  December  2015,  DOT&E  published  an  assessment  of 
the  results  of  operational  testing  of  the  Freedom  variant 
seaframe  and  SUW  mission  package  (Increments  1  and  2). 

Seaframes 

•  Freedom  variant: 

-  The  Navy  conducted  a  TSST  in  USS  Fort  Worth  (LCS  3) 
from  September  29,  2014  through  October  3,  2014,  in 
accordance  with  the  DOT&E-approved  trial  plan. 

-  In  November  2014,  LCS  3  deployed  for  extended 
operations  in  the  Western  Pacific  with  an  Increment  2 
SUW  mission  package  and  an  aviation  detachment  that 
included  an  MF1  60R  helicopter  and  an  MQ-8B  Fire 
Scout  VTUAV.  The  Navy  expects  LCS  3  to  return  to  her 
homeport  in  3QFY16. 


-  In  November  2015,  the  Navy  placed  USS  Milwaukee 
(LCS  5)  in  commission. 

•  Independence  variant: 

-  In  October  2014,  USS  Independence  (LCS  2)  hosted  a 
scheduled  phase  of  developmental  testing  focused  on 
integrated  seaframe  and  Increment  1  MCM  mission 
package  operations. 

-  In  January  2015,  the  Navy  conducted  developmental 
testing,  including  gunnery  events,  using  LCS  2.  The  ship 
then  sailed  from  San  Diego,  California,  to  the  Gulf  of 
Mexico,  arriving  in  Pensacola,  Florida,  on  February  17. 
Following  installation  and  grooming  of  the  Increment  1 
MCM  mission  package,  LCS  2  conducted  crew  training 
in  MCM  operations  in  preparation  for  TECHEVAL  of 
the  Independence  variant  LCS  and  Increment  1  MCM 
mission  package. 

-  From  May  through  August  2015,  the  Navy  conducted 
developmental  testing,  including  TECHEVAL,  of  the 
Independence  variant  seaframe  and  Increment  2  SUW 
mission  package  aboard  LCS  4.  This  TECHEVAL 
integrated  the  test  objectives  of  both  the  developmental 
and  operational  test  communities.  DOT&E  and  the 
Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  are  using  the  resulting  data  to  supplement 
data  collected  during  a  subsequent  operational  test. 
DOT&E  approved  an  operational  test  supplement  to 

the  developmental  test  plans,  and  DOT&E  personnel 
observed  the  testing  aboard  LCS  4. 

-  In  June  and  July  2015,  COTF  conducted  the 
cybersecurity  CVPA  phase  of  Operational  Test  C2 
(OT-C2)  of  the  Independence  variant  LCS  and  the 
Increment  1  MCM  mission  package  aboard  LCS  2 
while  the  ship  was  moored  in  Pensacola,  Florida.  The 
operational  testing  was  conducted  in  accordance  with  the 
test  plan  approved  by  DOT&E.  COTF  plans  to  complete 
the  final  phase  of  LCS  2  and  MCM  mission  package 
operational  cybersecurity  testing  and  all  other  OT-C2 
events  during  FY16. 

-  In  August  2015,  the  Navy  conducted  the  first  shipboard 
live  firing  of  the  ship’s  SeaRAM  system  against  a 
subsonic  aerial  drone.  The  Navy  had  attempted  to 
conduct  the  test  event  in  June,  but  had  to  postpone  the 
event  due  to  seaframe  equipment  failures.  The  Navy  had 
originally  planned  to  conduct  non-firing  tracking  runs 
against  aerial  drones,  but  these  events  were  canceled 
because  of  the  range  safety  restrictions  for  a  manned 
ship  that  preclude  conducting  such  test  events  with 
realistic  geometries.  The  live  fire  demonstration  was 
not  designed  to  be  an  operationally  realistic  test  of  the 
ship’s  capability.  The  aerial  drone  flight  profile  and 
configuration  were  not  threat  representative. 

-  In  August  and  September  2015,  the  Navy  conducted  the 
first  phase  of  operational  testing  of  the  Independence 
variant  seaframe  and  Increment  2  SUW  mission  package 
(Operational  Test  C4)  aboard  LCS  4.  Operational  testing 
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was  conducted  in  accordance  with  a  DOT&E-approved 
test  plan.  That  testing  consisted  of  an  examination  of  the 
seaframe’s  electronic  warfare  capability;  several  surface 
self-defense  events  against  small  boats  (without  the 
mission  package);  seaframe  evaluations  of  endurance, 
sprint  speed,  and  small  boat  launch  and  recovery  for 
Visit,  Board,  Search,  and  Seizure  missions  of  state.  The 
testing  also  examined  the  ship’s  ability,  when  equipped 
with  an  Increment  2  SUW  mission  package,  to  combat  a 
small  swarm  of  FIAC. 

-  Because  of  changes  to  the  ship’s  air  defense  system, 
SeaRAM,  and  additional  modifications  to  the  ship’s 
combat  system  and  networks,  a  second  phase  of 
operational  testing  of  the  Increment  2  version  of  the 
SUW  mission  package  and  Independence  variant 
seaframe  will  occur  in  3QFY16,  which  will  examine  the 
air  warfare  capabilities  of  the  seaframe,  cybersecurity 
upgrades,  and  the  remaining  SUW  events. 

-  USS  Jackson  (LCS  6)  completed  acceptance  trials  in 
June  2015;  the  Navy  accepted  delivery  in  August  2015 
and  placed  the  ship  in  commission  in  December  2015. 

SUW  Mission  Package 

•  During  3Q  and  4QFY15,  the  Navy  conducted 
developmental  testing  of  the  Increment  2  SUW  mission 
package  aboard  LCS  4. 

•  In  August  and  September  2015,  the  Navy  conducted 
operational  testing  of  the  Increment  2  SUW  mission 
package  aboard  LCS  4.  This  phase  of  the  operational 
test  examined  the  Independence  variant’s  self-defense 
capability  against  small  swarms  of  high-speed  boats  and  its 
effectiveness  for  Maritime  Security  Operations  requiring 
the  crew  to  intercept  and  board  a  vessel  suspected  of 
transporting  contraband  when  equipped  with  the  Increment 
2  SUW  mission  package.  The  testing  was  conducted  in 
accordance  with  a  DOT&E-approved  test  plan. 

•  COTF  conducted  a  shore-based  Quick  Reaction  Assessment 
of  an  MQ-8B  Fire  Scout  VTUAV  equipped  with  the 
AN/ZPY-4(1)  radar  in  May  and  June  2015.  The  Navy’s 
original  plans  for  the  Increment  2  MCM  mission  package 
called  for  the  MC-8B  VTUAV,  but  those  plans  are  now 

in  doubt.  The  Navy  plans  to  embark  the  larger  MQ-8C 
VTUAV  with  the  SUW  mission  package  starting  with 
Increment  3,  but  initial  plans  do  not  call  for  the  aircraft  to 
be  equipped  with  radar.  COTF  conducted  a  land-based 
operational  assessment  of  the  MQ-8C  in  November  2015, 
the  results  of  which  are  not  yet  available. 

MCM  Mission  Package 

•  During  1QFY15,  the  Navy  completed  the  last  scheduled 
phase  of  the  Increment  1  MCM  mission  package 
developmental  test  DT-B2  aboard  LCS  2. 

•  Having  completed  the  land-based  phase  of  an  operational 
assessment  of  the  AMNS  in  3QFY14  with  the  MH-60S 
helicopter  operating  from  Naval  Air  Station,  Oceana, 
Virginia,  the  Navy  conducted  the  ship-based  phase  of  the 
operational  assessment  aboard  LCS  2  in  1QFY15  during 
Increment  1  MCM  mission  package  developmental  testing. 


The  ship-based  phase  focused  on  shipboard  integration  and 
the  system’s  operational  suitability,  but  was  also  able  to 
collect  limited  effectiveness  data. 

•  The  Navy  also  completed  the  ship-based  phase  of  an 
Airborne  Laser  Mine  Detection  Systems  (ALMDS) 
operational  assessment  in  1QFY15  aboard  LCS  2  during 
Increment  1  MCM  mission  package  developmental 
testing.  The  test  collected  limited  data  to  examine  system 
effectiveness  and  the  shipboard  suitability  of  the  MH-60S 
helicopter  equipped  with  the  ALMDS. 

•  The  Navy  canceled  a  scheduled  operational  assessment  of 
Coastal  Battlefield  Reconnaissance  and  Analysis  (COBRA) 
Block  1  after  a  NASAAntares  rocket  exploded  just  after 
lift-off  from  the  Wallops  Island,  Virginia,  launch  pad  on 
October  28,  2014.  Although  all  test  preparations  had  been 
completed,  both  MQ-8B  Fire  Scout  VTUAVs  that  were  to 
host  the  COBRA  system  during  the  test  suffered  shrapnel 
damage  from  the  rocket  explosion.  In  December  2014, 
DOT&E  returned  the  Navy’s  revised  COBRA  Block  1 
TEMP  for  rework,  noting  that  the  schedule,  test  strategies, 
funding  profile,  and  planned  resources  no  longer  reflected 
the  state  of  the  program  following  cancelation  of  the 
operational  assessment. 

•  The  Navy  conducted  shore-based  developmental  testing 
(DT-B 1)  of  the  RMS,  consisting  of  the  v6.0  RMMV  and 
AN/AQS-20A/B  from  the  contractor’s  facility  at  West  Palm 
Beach,  Florida.  The  Navy  commenced  testing  in  December 
2014  with  an  upgraded  version  of  the  sensor,  designated 
AN/AQS-20B,  but  in  January  2015,  the  Navy  determined 
the  new  sensor  was  not  yet  sufficiently  mature  and  elected 
to  complete  testing  with  the  AN/AQS-20A  sonar.  The 
Navy  subsequently  suspended  testing  in  January  2015 

to  investigate  RMMV  reliability  problems  and  complete 
corrective  maintenance.  The  Navy  resumed  and  completed 
testing  in  March  2015. 

•  From  April  through  August  2015,  the  Navy  conducted 
TECHEVAL  of  the  Independence  variant  LCS  and 
Increment  1  MCM  mission  package  aboard  LCS  2. 
Although  the  Navy  originally  planned  to  conduct  the  test 
from  April  through  June  2015,  problems  with  failures 
of  seaframe  and  MCM  systems  caused  the  testing  to  be 
extended.  The  Navy  chose  to  extend  the  testing  further, 
conducting  another  evolution  of  the  MCM  scenario,  in 
order  to  provide  confidence  in  the  capabilities  of  the  ship 
and  mission  package  prior  to  entering  the  operational 
test  period.  Although  this  testing  was  developmental  in 
nature,  the  test  was  designed  to  integrate  the  objectives 
of  both  developmental  and  operational  test  communities. 
DOT&E  personnel  observed  the  testing  aboard  LCS  2.  If 
the  Navy  elects  to  continue  with  the  same  system  hardware 
and  software  configurations,  DOT&E  and  COTF  will  use 
the  resulting  data  to  supplement  data  collected  during 

the  operational  test.  If  the  Navy  decides  to  go  forward  to 
operational  testing  with  a  new  system,  integrated  test  data 
collected  in  FY15  may  not  be  representative  of  the  system 
the  Navy  intends  to  field,  and  the  Navy  might  need  to  repeat 
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some  portions  of  previous  tests  to  provide  the  requisite  data. 
Although  the  Navy  planned  to  complete  operational  testing 
of  the  Increment  1  MCM  mission  package  in  FY15,  only 
the  cybersecurity  CVPA  was  completed.  The  Navy  has 
delayed  the  remaining  OT-C2  events,  and  they  are  unlikely 
to  be  conducted  before  the  spring  of  2016,  at  the  earliest. 

•  In  an  August  2015  memorandum,  DOT&E  advised  the 
USD(AT&L)  that  the  reliability  of  the  RMS  and  its  RMMV 
poses  a  significant  risk  to  the  planned  operational  test  of 
the  Independence  variant  LCS  and  the  Increment  1  MCM 
mission  package  and  to  the  Navy’s  plan  to  field  and  sustain 
a  viable  LCS-based  minehunting  and  mine  clearance 
capability  prior  to  FY20.  DOT&E  recommended  that 

the  acquisition  strategy  for  these  systems  be  reexamined 
to  ensure  that  sufficient  testing  is  performed  to  inform 
the  procurement  of  additional  vehicles  and  cautioned 
that  continued  development  of  this  program  without  a 
fundamental  change  would  be  unlikely  to  result  in  a  system 
that  is  effective  and  suitable. 

•  In  September  2015,  the  Navy  chartered  an  independent 
program  review  of  the  RMS,  including  an  evaluation  of 
potential  alternative  MCM  systems.  Their  report  is  due 
in  late  1QFY16.  Additionally,  USD(AT&L)  delayed  its 
review  to  consider  approval  to  restart  RMS  low-rate  initial 
production  until  at  least  3QFY16. 

•  In  November  2015,  DOT&E  provided  the  USD(AT&L),  the 
Assistant  Secretary  of  the  Navy  for  Research  Development 
and  Acquisition,  and  the  Program  Executive  Officer  for 
Littoral  Combat  Ships  a  classified  assessment  of  the 
performance  of  the  Independence  variant  seaframe  and 
Increment  1  MCM  mission  package.  DOT&E  based  the 
assessment  on  the  data  collected  during  the  TECHEVAL 
and  earlier  periods  of  development  and  operational  testing. 

•  Also  in  November  2015,  DOT&E  provided  comments  to 
the  Joint  Staff  on  the  Navy’s  draft  Capability  Production 
Document  for  the  “Phase  1”  (formerly  Increment  1)  MCM 
mission  package. 

ASW  Mission  Package 

•  The  Navy  did  not  conduct  any  at-sea  testing  of  the  ASW 
mission  package  in  FY 1 5  due  to  limited  ship  availability 
and  changes  to  the  system’s  design.  The  Navy  continued  its 
efforts  on  a  weight  reduction  program  for  the  components 
of  the  mission  package,  including  the  handling  system 

and  support  structures  for  the  variable  depth  sonar  and 
multi-function  towed  array. 

Assessment 

This  assessment  is  based  on  information  from  post-delivery 
test  and  trial  events,  fleet  operations,  developmental  testing, 
results  provided  by  the  Navy  Program  Offices,  operational 
assessments  of  MCM  mission  systems,  operational  testing  of 
the  Independence  variant  seaframe  with  the  Increment  2  SUW 
mission  package,  and  operational  cybersecurity  testing  conducted 
in  LCS  2.  A  summary  of  DOT&E’s  December  2015  report  on 
the  Freedom  variant  equipped  with  the  Increment  2  SUW  mission 
package  is  also  provided  below. 


Program 

•  The  Navy  intends  to  field  LCS  capabilities  incrementally 
as  mission  package  systems  mature  and  become  ready 
for  fleet  use.  Since  the  Navy  expects  each  increment  to 
deliver  significant  increases  in  mission  capability,  the 
approved  TEMP  calls  for  an  appropriately-designed  phase 
of  OT&E  on  all  delivered  mission  package  increments  on 
each  seaframe  variant.  However,  because  the  content  of  the 
later  increments  is  not  yet  final,  the  details  of  the  testing  to 
be  accomplished  for  later  increments  of  mission  package 
capability  are  yet  to  be  planned. 

-  Initial  phases  of  operational  testing  were  completed  in 
FY14  for  the  Freedom  variant  seaframe  and  Increment  2 
SUW  mission  package  and  partially  completed  in  FY15 
for  the  Independence  variant  seaframe  and  Increment  2 
SUW  mission  package  embarked  on  that  variant.  The 
final  phases  of  operational  testing  will  not  be  completed 
until  the  full  mission  package  capability  is  available. 

The  Navy  expects  to  complete  those  final  phases  of 
operational  testing  in  the  FY  1 8  timeframe,  depending 
on  the  decision  whether  to  pursue  an  Increment  4  of 
the  SUW  mission  package.  It  is  unknown  when  either 
the  MCM  mission  package  or  ASW  mission  package 
operational  test  programs  will  be  complete. 

-  The  Navy  is  finding  it  difficult  to  follow  the  plan  in 
the  approved  TEMR  The  integration  of  concurrently 
developed  components  into  the  MCM  mission  package 
has  not  been  as  easy  as  originally  planned,  and  the 
Navy  has  appropriately  decided  to  conduct  additional 
developmental  testing  after  making  system  changes 

in  an  attempt  to  correct  the  identified  problems  with 
subsystem  performance.  Decisions  to  include  the  ships 
in  major  fleet  exercises  and  to  press  for  establishment  of 
a  continuous,  multi-LCS  presence  overseas  in  FY17  are 
also  reducing  the  number  of  ships  available  to  participate 
in  the  test  program.  The  Navy  is  challenged  to  meet  the 
simultaneous  demands  for  LCS  fleet  operations,  both 
forward  deployed  and  in  home  waters,  as  well  as  mission 
package  development  and  the  necessary  developmental 
and  operational  testing. 

•  Additionally,  the  Navy  directed  changes  to  the  seaframe 
designs  based  on  the  results  of  early  developmental  testing 
and  operations.  The  Navy  has  indicated  that  the  seaframe 
designs  will  be  stabilized  in  the  third  ship  of  each  variant 
(LCS  5  and  LCS  6). 

Seaframes 

•  In  the  report  to  Congress  responding  to  the  FY15  NDAA, 
DOT&E  noted  that  the  envisioned  missions,  use  of 
unmanned  vehicles,  and  operating  environments  have 
shifted  relative  to  the  original  LCS  vision.  DOT&E 
concluded  that  the  use  of  LCS  as  a  forward-deployed 
combatant,  where  it  might  be  involved  in  intense  naval 
conflict  as  now  intended,  appears  to  be  inconsistent  with 
its  inherent  survivability  in  those  same  environments.  The 
ability  of  LCS  to  successfully  execute  significant  aspects 
of  the  envisioned  CONOPS  depends  on  the  effectiveness 
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of  the  mission  packages.  To  date,  the  Navy  has  not  yet 
demonstrated  effective  capability  for  either  the  MCM  or 
the  AS  W  mission  package.  The  Increment  2  SUW  mission 
package  has  demonstrated  some  modest  ability  to  aid  the 
ship  in  defending  itself  against  small  swarms  of  FIAC,  and 
the  ability  to  conduct  maritime  security  operations. 

•  While  both  seaframe  variants  are  fast  and  highly 
maneuverable,  they  are  lightly  armed  and  were  not  designed 
to  provide  any  significant  offensive  capability  without 

the  planned  Increment  4  SUW  mission  package  or  the 
Increment  2  ASW  mission  package.  In  comparison  to  other 
Navy  ships,  the  LCS  seaframes  have  relatively  modest 
air  defense  capabilities  that  cannot  be  characterized  fully 
until  planned  tests  on  LCS  7  and  LCS  8  and  the  Navy’s 
unmanned  self-defense  test  ship  provide  data  for  the  Navy 
P  high-fidelity  modeling  and  simulation  analyses.  The 
Navy  plans  to  begin  those  tests  in  FY17.  In  FY15,  DOT&E 
learned  that  PEO  IWS  stopped  work  on  the  P|;A  Test  Bed 
for  the  Freedom  variant  because  the  high-fidelity  model 
of  the  ship’s  AN/SPS-75  radar  was  not  being  developed. 
Development  of  an  acceptable  radar  model  requires 
intellectual  property  rights  that  the  Navy  does  not  hold  and 
is  not  actively  seeking.  Although  less  critical  because  of  the 
combat  system  architecture  of  the  Independence  variant,  the 
Navy  has  also  been  unable  to  develop  a  high-fidelity  model 
of  that  ship’s  AN/SPS-77  radar  for  the  same  reason.  In  an 
August  2015  memorandum,  DOT&E  advised  Navy  officials 
that  the  lack  of  these  radar  models  threatens  the  viability 
of  the  Navy’s  strategy  for  evaluation  of  LCS  air  defense 
capabilities  and  suggested  alternative  strategies  specific 
to  each  seaframe  variant.  The  alternative  test  strategies 
suggest  additional  live  testing  that  might  be  acceptable. 
Near-term  resolution  will  be  required  to  avoid  delaying  PRA 
Test  Bed  analyses  needed  to  finalize  DOT&E’s  evaluation 
of  LCS  air  defense  effectiveness.  The  Navy  has  not  decided 
what  course  of  action  they  want  to  pursue. 

•  Neither  LCS  variant  has  been  operationally  tested  to 
evaluate  its  effectiveness  against  unmanned  aerial  vehicles 
and  slow-flying  aircraft.  Although  the  Navy  had  planned 
to  test  the  Independence  variant’s  capability  to  defeat 
such  threats  in  FY15,  the  testing  was  canceled  because 

of  range  safety  requirements  that  would  have  precluded 
operationally  realistic  testing.  DOT&E  concurred  with  this 
decision  because  proceeding  with  an  unrealistic  test  would 
have  been  a  needless  waste  of  resources. 

•  The  seaframes  include  no  systems  designed  to  counter 
torpedo  attacks  or  detect  and  avoid  mines  without  the 
appropriately  configured  mission  packages  installed. 

•  Crew  size  limits  the  mission  capabilities,  combat 
endurance,  maintenance  capacity,  and  recoverability  of 
the  ships.  The  Navy  continues  to  review  LCS  manning  to 
determine  appropriate  levels  and  has  added  20  berths  to  all 
seaframes.  The  increased  berthing  supports  small  increases 
in  the  size  of  the  core  crew,  mission  package  and  aviation 
detachments,  but  still  leaves  the  ships  heavily  dependent 


on  Navy  shore  organizations  for  administrative  and 

maintenance  support. 

•  Freedom  Variant  Seaframe  (LCS  1  and  3): 

-  Although  not  all  aspects  of  operational  effectiveness 
and  suitability  could  be  examined  during  the  2014 
operational  test,  that  testing  identified  shortcomings 
in  cybersecurity,  air  defense,  surface  self-defense, 
reliability,  maintainability,  speed  and  endurance,  air 
operations,  and  other  operations. 

-  Cybersecurity.  Cybersecurity  testing  conducted 
aboard  LCS  3  uncovered  significant  deficiencies  in  the 
ship’s  capability  to  protect  the  security  of  information 
and  prevent  malicious  intrusion.  Many  of  these 
deficiencies  were  previously  discovered  during  the 
2012  Quick  Reaction  Assessment  that  COTF  conducted 
in  USS  Freedom  (LCS  1).  Although  the  Navy  is 
developing  plans  to  modify  the  network  architecture  in 
the  Freedom  variant  ships  to  enhance  cybersecurity,  the 
severity  of  the  cybersecurity  problems  will  degrade  the 
operational  effectiveness  of  Freedom  variant  seaframes 
until  the  problems  are  corrected. 

-  Air  Defense.  Aircraft  tracking  events  conducted  during 
operational  testing  aboard  LCS  3  demonstrated  that 

the  crew  was  unable  to  detect  and  track  some  types  of 
air  threats  well  enough  to  engage  them.  The  inability 
to  engage  these  air  threats  leaves  the  ship  without  an 
effective  air  defense  in  some  situations.  As  expected, 
tracking  performance  improved  significantly  when 
the  LCS  received  tracking  information  via  datalink 
from  a  nearby  Aegis  destroyer.  Since  the  radar  had 
demonstrated  significantly  better  tracking  performance 
during  the  Navy’s  TECHEVAL,  when  subject  matter 
experts  were  embarked  to  advise  and  train  the  crew,  it  is 
possible  that  the  crew’s  lack  of  proficiency  in  the  use  of 
the  radar’s  controls  during  the  initial  test  contributed  to 
the  poor  performance. 

-  The  lack  of  integration  between  the  WBR-2000 
Electronic  Support  Measures  (ESM)  system  and  the 
RAM  system  limits  the  ship’s  capability  to  make  best  use 
of  its  limited  RAM  inventory.  The  inability  to  provide 
electronic  signal  measurements  to  RAM  can  reduce  the 
likelihood  that  some  of  the  missiles  fired  will  acquire  and 
home  on  the  target,  thus  reducing  the  probability  that  the 
ship  will  be  able  to  defeat  an  incoming  raid  of  ASCMs. 

-  Surface  Self  Defense.  LCS  3  demonstrated  the 
seaframe’s  core  capability  for  self-defense  against  a 
small  boat  during  two  trials  conducted  under  favorable 
conditions,  but  the  operational  test  did  not  include 
enough  trials  to  determine  whether  a  Freedom  variant 
LCS  can  defeat  such  a  threat  with  regularity.  Testing 
was  not  conducted  in  a  realistic  cluttered  environment 
where  identification  of  threats  will  be  more  challenging. 
Although  the  Navy  attempted  to  collect  additional 
data  on  the  core  seaframe’s  performance  from  swarm 
presentations,  DOT&E  determined  that  the  data  were 


232  LCS 


F Y1  5  NAVY  PROGRAMS 


invalid.  The  57  mm  gun  failed  to  achieve  a  mission  kill 
during  one  swarm  presentation,  and  the  target  killed  by 
the  57  mm  gun  during  a  second  swarm  presentation  had 
previously  been  engaged  by  the  SUW  mission  package’s 
30  mm  guns.  The  57  mm  gun  itself  performed  reliably 
during  the  operational  test,  but  the  DORNA  EOD  system 
used  to  target  the  gun  experienced  numerous  laser  faults 
that  interrupted  some  engagements  and  reduced  the 
ship’s  effectiveness  against  attacking  small  boats.  An 
inopportune  fault  could  allow  an  attacker  to  close  within 
his  weapon  range.  The  LCS  3  crew  did  not  attempt  to 
use  the  ship’s  AN/SPS-75  ASR  for  gun  targeting  during 
the  operational  test. 

-  Missions  of  State.  Operational  testing  confirmed  earlier 
observations  that,  except  for  the  ships’  lack  of  fuel 
endurance,  the  Freedom  variant  is  suited  for  Maritime 
Security  Operations.  LCS  3  readily  demonstrated  the 
capability  to  position,  launch,  and  recover  the  11-meter 
boats  included  in  the  SUW  mission  package  when  the 
launch,  recovery,  and  handling  system  is  operational. 

-  Speed  and  Endurance.  During  operational  testing, 

LCS  3  did  not  demonstrate  that  it  could  achieve  the 
Navy  requirement  for  fuel  endurance  (operating  range) 
at  the  prescribed  transit  speed  or  at  sprint  speed.  Based 
on  fuel  consumption  data  collected  during  the  test,  the 
ship’s  operating  range  at  14.4  knots  (the  ship’s  average 
speed  during  the  trial)  is  estimated  to  be  approximately 
1,960  nautical  miles  (Navy  requirement:  3,500  nautical 
miles  at  14  knots)  and  the  operating  range  at  43.6  knots 
is  approximately  855  nautical  miles  (Navy  requirement: 
1,000  nautical  miles  at  40  knots).  In  an  emergency, 
the  ship  could  use  its  aviation  fuel  (F-44)  to  extend  the 
transit  and  sprint  ranges  by  360  and  157  nautical  miles, 
respectively.  The  shortfall  in  endurance  may  limit  the 
flexibility  of  the  ship’s  operations  in  the  Pacific  and  place 
a  heavier  than  anticipated  demand  on  fleet  logistics.  The 
Navy’s  report  from  calm  water  trials  suggests  that  the 
ship  can  achieve  an  endurance  range  of  3,500  nautical 
miles  at  an  average  (but  not  constant)  speed  of  14  knots 
by  using  a  more  economical  propulsion  configuration 
(two  propulsion  diesel  engines  and  two  steerable  water 
jets).  The  ship  cannot  attain  a  speed  of  14  knots  in  this 
configuration  when  fully  loaded  with  fuel. 

-  Aircraft  Operations.  The  Freedom  variant  LCS 
has  sufficient  aviation  facilities  and  meets  Navy 
requirements  to  safely  launch,  recover,  and  handle 
the  MH-60R  helicopter  while  operating  in  up  to  Sea 
State  4  conditions.  However,  the  ship  frequently  had 
trouble  establishing  and  maintaining  a  Tactical  Common 
Data  Link  (TCDL)  with  the  aircraft  during  the  FY14 
operational  test.  The  crew’s  efforts  were  hampered  by  an 
antenna  failure  and  the  lack  of  technical  documentation 
on  the  operation  and  maintenance  of  the  datalink. 

The  TCDL  is  the  primary  conduit  for  sharing  tactical 
information,  including  voice  reports,  radar  tracks,  and 
radar  and  electro-optical  and  infrared  (EO/IR)  sensor 
video  between  the  MH-60R  helicopter  and  the  LCS. 


-  Other  Operations.  COTF  exercised  LCS  3  and  her 
crew  in  a  variety  of  other  shipboard  evolutions  during 
an  operational  test,  including  anti-terrorism/force 
protection,  damage  control,  mooring  and  unmooring, 
navigation,  refueling  at  sea,  vertical  replenishment, 
man-overboard  recovery,  and  communications.  These 
evolutions  yielded  no  quantitative  data;  COTF  evaluated 
the  ship’s  performance  qualitatively.  Except  as  noted 
below,  DOT&E  observers  reported  that  the  ship’s 
performance  during  the  observed  evolutions  was 
consistent  with  the  Navy’s  expectations  for  any  surface 
combatant. 

■  The  anchoring  system  could  not  securely  anchor  the 
ship  in  an  area  with  a  bottom  composed  of  sand  and 
shells.  On  several  occasions,  the  ship  was  unable 
to  set  the  anchor  despite  repeated  efforts.  It  appears 
that  the  anchor  and  chain  are  too  light  and  there  is  too 
much  friction  along  the  anchor  chain’s  internal  path 
from  the  chain  locker  to  the  hawse  pipe  to  allow  the 
anchor  and  chain  to  pay  out  smoothly.  Inability  to 
anchor  the  ship  securely  could  force  the  ship  to  remain 
at  sea  when  anchoring  would  be  preferred  and  could 
hazard  the  ship  if  it  loses  power  in  coastal  waters  or 
encounters  other  circumstances  where  anchoring  is 
required. 

■  The  fenders  designed  to  guide  the  1 1 -meter  Rigid 
Hull  Inflatable  Boats  included  in  the  SUW  mission 
package  during  launch  and  recovery  are  fragile  and 
occasionally  sheared  off  when  impacted  by  the  boats 
during  operational  testing.  Although  the  fenders  had 
undergone  several  redesigns,  they  were  not  yet  strong 
enough  to  sustain  such  impacts.  Loss  of  one  or  more 
of  the  fenders  could  delay  or  preclude  boat  launch  and 
recovery  needed  to  support  Visit,  Board,  Search,  and 
Seizure  operations. 

-  Operational  Suitability.  The  Freedom  variant  LCS 
seaframe  is  not  operationally  suitable  because  many 
of  its  critical  systems  supporting  ship  operations,  core 
mission  functions,  and  mission  package  operations  are 
unreliable;  and  the  ship’s  crew  does  not  have  adequate 
training,  tools,  and  technical  documentation  to  diagnose 
failures  or  correct  them  when  they  occur.  By  design, 
the  ship’s  small  crew  does  not  have  the  capacity  to 
effect  major  repairs.  Instead,  the  Navy’s  support 
concept  depends  on  the  use  of  remote  assistance  in 
troubleshooting  problems  and  the  use  of  Navy  repair 
organizations  and  contractors  for  repair  assistance. 
However,  the  Navy’s  limited  stock  of  repair  parts 

for  LCS  systems,  many  of  which  were  sourced  from 
offshore  vendors,  can  result  in  long  logistics  delays  and 
occasionally  forces  the  Navy  to  resort  to  cannibalization 
of  another  ship  in  order  to  expedite  repairs. 

■  The  FY14  operational  test  did  not  yield  sufficient 
evidence  to  report  whether  the  mission  critical 
components  were  individually  meeting  the  Navy’s 
reliability  thresholds;  the  combined  data  for  all  of 
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the  components  revealed  the  aggregate  reliability  of 
Propulsion  and  Maneuvering  and  Navigation  and  Ship 
Control  functional  areas  were  extremely  low. 

■  The  aggregate  reliability  of  the  components 
that  comprise  the  core  mission  area  (e.g.,  total 
ship  computing  environment,  air  search  radar, 
electro-optical  tracking  system,  and  electronic  support 
measures)  was  also  poor.  Based  on  the  operational 
test  results,  the  probability  of  successfully  completing 
a  30-day  mission  without  a  critical  failure  of  a  core 
mission  subsystem  that  reduces  the  ship’s  full  mission 
capability  is  less  than  5  percent. 

■  The  aggregate  reliability  of  the  mission  package 
support  functional  area  (mission  package  support 
systems,  mission  package  computing  environment, 
waterborne  mission  equipment,  and  airborne  mission 
equipment)  was  somewhat  better  than  that  of  other 
functional  areas  but,  at  0.38,  still  well  below  the 
Navy’s  reliability  threshold  (0.9). 

■  Low  reliability,  maintenance  challenges,  and  logistics 
delays  reduced  LCS  3 ’s  operational  availability  for 
Mobility  (Propulsion  and  Maneuvering),  Total  Ship 
Computing  Environment  (TSCE),  Seaframe  Sensors 
and  Controls,  Communications,  and  Mission  Package 
Support  to  below  the  Navy’s  threshold  requirement 
(0.85).  Failures  of  the  Propulsion  and  Maneuvering 
subsystems  and  the  TSCE,  which  are  fundamental  to 
ship  operations,  caused  the  ship  to  return  to  port  for 
repairs  or  reduced  readiness  while  at  sea  for  42  and 
36  days,  respectively.  The  demonstrated  availability 
of  six  other  mission-critical  subsystems  was  above  the 
Navy’s  threshold:  Engineering  Controls,  Navigation 
and  Ship  Control,  Electrical  Power  Generation  and 
Distribution,  Auxiliary  Systems,  Damage  Control, 
and  Seaframe  Engagement  Weapons.  The  LCS  3 
seaframe  was  partially  or  fully  mission  capable  just 
over  60  percent  of  the  time  in  Air  Warfare  and  nearly 
85  percent  of  the  time  in  Surface  Warfare,  but  partial 
mission  capability  can  result  in  a  significant  reduction 
in  operational  effectiveness. 

•  Independence  Variant  Seaframe  (LCS  2  and  4): 

-  DOT&E  is  still  analyzing  data  on  the  performance 
of  the  Independence  variant  seaframe.  During  the 
period  under  review,  LCS  2  underwent  developmental 
testing  and  TECFIEVAL  with  the  Increment  1  MCM 
mission  package  embarked,  as  well  as  the  first 
phase  of  operational  cybersecurity  testing  (CVPA). 
Additionally,  LCS  4,  with  the  Increment  2  SUW  mission 
package  embarked,  underwent  developmental  testing, 
TECHEVAL,  and  the  first  phase  of  planned  operational 
testing.  Observer  reports  and  preliminary  data  analyses 
provide  sufficient  evidence  of  numerous  Independence 
variant  seaframe  deficiencies  that  significantly  degrade 
the  ships’  operational  effectiveness  and  suitability.  Many 
of  these  deficiencies  are  detailed  below. 


-  Air  Defense.  The  Independence  variant  ships  are  the 
first  to  use  the  SeaRAM  air  defense  system.  Although 
SeaRAM  has  never  been  operationally  tested,  it  shares 
many  components  with  the  Phalanx  Close-In  Weapon 
System,  which  is  widely  installed  in  the  fleet  as  a 
secondary  or  tertiary  close-in  self-defense  system. 

The  Navy  completed  the  first  at-sea  demonstration 

of  the  SeaRAM  system  in  LCS  4  in  2015  during  an 
engagement  against  a  non-maneuvering,  subsonic 
aerial  target  (BQM-74)  with  radio  frequency  and 
infrared  augmentation  that  were  not  consistent  with  the 
characteristics  of  realistic  threats.  Because  SeaRAM  is 
a  self-contained  system  that  integrates  the  Phalanx  radar, 
track  processing,  and  ESM  receiver  it  should  provide  an 
air  defense  capability  on  par  with  other  RAM-equipped 
ships  in  the  fleet  as  long  as  the  AN/SPS-77  ASR  radar 
can  detect  the  incoming  threat(s)  and  the  crew  can 
maneuver  the  ship  to  place  the  threat(s)  in  SeaRAM’s 
engagement  zone.  However,  as  with  the  Freedom 
variant,  the  ship’s  air  defense  effectiveness  will  remain 
unproven  until  live  operational  testing  is  conducted  on 
a  manned  ship,  on  the  unmanned  self-defense  test  ship, 
and  using  an  appropriately  designed  PpA  Test  Bed.  That 
testing  is  scheduled  to  begin  in  3QFY16  aboard  the 
self-defense  test  ship  and  1QFY17  aboard  LCS  8.  The 
Navy  plans  to  complete  testing  utilizing  the  PR  x  Test  Bed 
in  FY18,  but  those  plans  are  in  doubt  due  to  issue  with 
the  radar  modeling  explained  earlier  in  this  report. 

-  Upon  learning  that  the  Navy  planned  to  upgrade  the 
SeaRAM  system  installed  in  LCS  4  to  bring  it  to  the 
same  configuration  as  the  system  being  installed  in 
Aegis  destroyers,  and  that  those  upgrades  and  other 
combat  system  upgrades  were  to  be  installed  in  1QFY16 
and  3QFY16,  DOT&E  recommended  that  some  of  the 
Independence  variant  air  warfare  operational  testing 
planned  to  complete  in  FY15  be  delayed  so  it  could  be 
conducted  with  the  ship  in  its  deployment  configuration. 
The  Navy  accepted  the  recommendation  and  now  plans 
to  conduct  the  air  warfare  tracking  events  in  late  FY16. 
The  Navy  plans  to  complete  live  SeaRAM  testing  on 
LCS  8  in  FY17. 

-  The  Program  Office  conducted  several  developmental 
test  events  to  evaluate  the  ship’s  capability  to  detect, 
track,  and  engage  so-called  Low  Slow  Flyers  (LSF) 
(unmanned  aerial  vehicles,  slow-flying  fixed-wing 
aircraft,  and  helicopters)  in  mid-2015.  The  only  sensor 
used  to  provide  tracking  information  for  engaging 
LSFs  with  the  57  mm  gun  is  the  SAFIRE  EO/IR 
system.  The  test  events  demonstrated  that  SAFIRE 
was  unable  to  provide  reliable  tracking  information 
against  some  targets.  Furthermore,  the  safety  standoff 
requirements  on  Navy  test  ranges  were  so  severe  as  to 
preclude  meaningful  live  fire  shooting  engagements. 
Because  of  these  constraints,  the  program  decided  to 
cancel  all  subsequent  live  fire  events,  conceding  that 
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the  Independence  variant  is  unlikely  to  be  successful 
consistently  when  engaging  some  LSFs  until  future 
upgrades  of  SAFIRE  can  be  implemented.  Live  firing 
events  planned  during  operational  testing  were  also 
canceled,  as  the  results  from  developmental  testing  were 
sufficient  to  conclude  that  the  Independence  variant  will 
not  likely  be  effective  in  these  scenarios  against  some 
LSFs.  Future  testing  against  LSFs  will  not  be  possible 
until  the  Navy  finds  a  solution  to  the  severe  safety 
constraints  that  preclude  engaging  realistic  targets. 

-  ESM  Testing.  While  most  air  warfare  testing  was 
delayed  to  FY16,  COTF  completed  testing  of  the 
Independence  variant’s  ES-3601  ESM  system  during 
the  FY15  operational  test.  COTF  used  Lear  aircraft 
equipped  with  ASCM  seeker  simulators  to  represent  the 
ASCM  threats.  Although  DOT&E  analysis  of  the  test 
data  is  not  complete,  DOT&E  observed  that  the  ES-3601 
detected  the  presence  of  the  ASCM  seekers  in  most 
instances  but  did  not  reliably  identify  certain  threats. 

-  Surface  Self-Defense.  The  Independence  variant 
seaframe’s  surface  self-defense  effectiveness  was  tested 
during  developmental,  integrated,  and  operational  test 
firing  events  in  2015.  These  events  tested  the  crew’s 
capability  to  defeat  a  single  small  boat  using  the 
seaframe’s  57  mm  gun.  DOT&E  considered  three  of 
the  developmental  test  events  as  sufficient  to  provide 
data  for  the  operational  effectiveness  determination, 

in  addition  to  the  two  dedicated  operational  test  events 
for  surface  self-defense.  Prior  to  these  five  events,  the 
Navy  also  conducted  three  additional  developmental 
test  events,  which  revealed  gun  faults  and  fuzing 
errors.  The  program  corrected  these  problems  before 
proceeding  to  the  integrated  and  operational  test  events. 
LCS  4  successfully  defeated  the  attacking  boat  with  the 
seaframe’s  MK  110  57  mm  gun  system  during  four  of 
the  five  presentations  considered  either  integrated  or 
operational  test  events.  The  firing  presentations  were 
judged  successful  if  a  “mission  kill’’  or  “mobility  kill” 
was  achieved  before  the  attacker  could  approach  within 
the  effective  range  of  its  weapon(s)  -  the  prescribed 
“keep-out”  range.  Since,  in  the  test  environment,  the 
attacker  was  the  only  boat  in  the  area,  it  was  easily 
classified  as  a  threat  well  beyond  the  effective  range  of 
the  ship’s  weapons.  The  Navy  has  not  conducted  any 
testing  to  determine  how  well  the  ship  will  perform  when 
faced  with  an  attack  in  a  realistic  cluttered  maritime 
environment  including  both  neutral  and  hostile  craft; 
the  Navy  has  also  not  conducted  operational  testing  to 
determine  how  well  the  ship  (without  the  SUW  mission 
package)  will  perform  against  multiple  attacking  boats. 

■  Two  of  the  surface  self-defense  failures  were  caused 
by  MK  110  57  mm  gun  malfunctions.  During  the 
first  presentation,  the  gun  operator’s  panel  displayed 
multiple  fault  indications,  and  the  operator  was 
unable  to  change  the  fuze  setting  from  proximity 
mode  to  the  recommended  point  detonation  (impact) 


mode.  Technicians  subsequently  determined  that  a 
gun  component  had  failed,  and  the  gun  was  repaired 
on  July  7,  2015.  The  second  presentation  on  July  18 
resulted  in  failure  when  the  57  mm  gun  loading 
mechanism  jammed  while  the  operator  was  attempting 
to  reload  the  gun.  With  the  assistance  of  a  civilian 
gun  system  technician,  the  crew  downloaded  the 
remaining  ammunition,  cleared  the  jam,  and  restored 
the  gun  to  “single-sided”  operation  in  about  4  hours 
by  consolidating  good  components.  Until  repaired 
on  August  7,  2015,  the  gun  was  limited  to  firing 
60  rounds  before  reloading.  Technical  issues  with 
SAFIRE  performance,  including  inability  to  track 
small  surface  craft  automatically  once  acquired 
(auto-track),  low  targeting  update  rate,  poor  bearing 
accuracy,  and  unwieldy  operator  interface  as  well 
as  persistent  problems  with  gun  system  accuracy 
resulted  in  excessive  ammunition  consumption  to 
achieve  these  modest  results.  The  testing  revealed 
that  although  successful  in  most  of  these  events,  had 
the  ship  been  required  to  engage  multiple  small  boats, 
the  crew  would  be  forced  to  reload  the  gun,  which 
could  interrupt  engagements.  Thus,  the  Independence 
variant  seaframe  will  be  challenged  to  defeat 
threat-representative  boat  swarms  in  an  operational 
environment  and  could  exhaust  its  supply  of  57  mm 
ammunition  if  faced  with  multiple  engagements. 

■  LCS  4  found  it  necessary  to  supplement  the  watch 
team  with  an  additional  watchstander  just  to  operate 
SAFIRE,  leaving  management  of  the  gun  to  a 
second  operator,  even  though  the  staffing  plan  calls  for 
one  operator  to  handle  both  functions.  The  small  LCS 
crew  does  not  include  enough  trained  operators  to 
maintain  this  watch  arrangement  for  any  appreciable 
length  of  time. 

■  Gun  accuracy  problems  have  been  observed  in  both 
LCS  2  and  LCS  4,  with  the  57  mm  gun  consistently 
firing  short  of  the  target  when  shooting  to  port  and 
beyond  the  target  when  shooting  to  starboard.  The 
Navy  has  not  yet  identified  the  root  cause  of  the 
problem  but  has  reduced  the  error  such  that  the 
operator  can  compensate  using  normal  procedures. 

■  On  one  occasion,  the  shock  caused  by  firing  the 
57  mm  gun  unseated  a  network  card,  disabling  the 
steering  controls  on  the  bridge  and  forcing  the  crew 
to  steer  the  ship  from  an  alternate  location.  On 
another  occasion,  gunfire  shook  network  cables 
loose,  disabling  several  combat  systems,  including 
the  AN/SPS-77  ASR  and  the  57  mm  gun.  While 
the  ship  was  able  to  recover  from  this  failure  within 
a  few  minutes  and  continue  the  engagement,  these 
interruptions  prolonged  the  ship’s  exposure  to  the 
advancing  threat  and  reduced  the  crew’s  situational 
awareness  during  the  repair.  Failures  of  this  nature 
demonstrate  the  need  for  full  ship  shock  trials,  which 
are  currently  planned  to  be  conducted  on  LCS  6. 
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-  Missions  of  State.  LCS  4  completed  six  mock  Missions 
of  State  during  OT-C4  requiring  the  launch  and  recovery 
of  two  11 -meter  Rigid  Hull  Inflatable  Boats.  LCS  4  met 
the  60-minute  launch  requirement,  but  on  average  was 
not  able  to  meet  the  60-minute  recovery  requirement. 
Faults  in  the  Twin-Boom  Extensible  Crane  (TBEC)  and 
problems  with  the  Surface  Tow  Cradle  were  responsible 
for  the  time  delays  during  recovery  operations.  The 
cumbersome  multi-step  boat  launch/recovery  process 
has  several  ‘single  points  of  failure’  that  increase  the 
likelihood  of  delays  and  the  possibility  of  mission 
failure,  including  the  Surface  Tow  Cradle,  TBEC,  the 
Mobicon  straddle  carrier,  and  a  forklift.  The  failure  of 
any  of  these  components  can  halt  boat  operations  and 
could  leave  a  boat  stranded  at  sea. 

-  Endurance  at  transit  speed.  LCS  4  demonstrated  that 
the  Independence  variant  seaframe’s  fuel  endurance  at  a 
transit  speed  of  14  knots  exceeds  the  Navy  requirement. 
Assuming  that  all  of  the  ship’s  “burnable”  F-76  fuel 
could  actually  be  consumed,  LCS  4  demonstrated  a 
fuel  endurance  of  5,345  nautical  miles  at  14  knots 
based  on  an  hourly  consumption  rate  of  42 1  gallons 
during  a  6-hour  trial.  In  reality,  no  ship  would  ever 
plan  to  consume  all  of  its  fuel  during  a  transit  because 
of  the  need  to  maintain  a  reserve  for  contingencies.  If 

a  20  percent  of  fuel  buffer  were  maintained,  the  ship’s 
endurance  would  be  4,242  nautical  miles. 

-  Sprint  speed  and  endurance.  COTF  reported  that 
LCS  4  demonstrated  an  average  sprint  speed  of 

37.9  knots  during  a  3-hour  trial  on  September  10  (Navy 
requirement:  40  knots).  Based  on  the  fuel  consumption 
rate  and  the  amount  of  practically  available  fuel,  an 
Independence  variant  ship  would  be  able  to  travel 
nearly  1,000  nautical  miles  in  25  hours  at  this  speed 
(Navy  requirement:  1,250  nautical  miles  at  40  knots). 
COTF  noted  that  the  ship  was  unable  to  maintain  the 
correct  trim  during  the  trial  because  the  interceptors 
(components  of  the  ride  control  system  designed  to  assist 
with  trim  control)  were  inoperative  and  that  the  crew  had 
to  change  five  fuel  oil  pre-filters  during  the  trial  to  keep 
the  gas  turbine  engines  on  line.  LCS  4  has  long-standing 
problems  with  her  ride  control  system  hardware, 
including  interceptors,  fins,  and  T-Max  rudders,  that 
affect  her  maneuverability.  The  ship  also  had  reported 
recurring  problems  with  frequent  clogging  of  the  gas 
turbine  engine  fuel  oil  conditioning  module  pre-filters 
and  coalescers,  and  found  it  difficult  to  maintain  high 
speed  for  prolonged  periods.  The  three-hour  trial 
conducted  on  September  10  was  reportedly  the  longest 
period  of  sustained  high-speed  operations  in  the  ship’s 
history. 

-  Aircraft  Operations.  Observers  reported  difficulties 
with  the  establishment  and  maintenance  of  the  Tactical 
Common  Data  Link  (TCDL),  an  encrypted  point-to-point 
datalink.  When  available,  the  TCDL  allows  transmission 
of  video,  data,  and  voice  communications  between  the 


aircraft  and  the  LCS.  However,  like  LCS  3,  LCS  4 
lacked  adequate  documentation  on  the  operation  and 
maintenance  of  TCDL  equipment.  Flight  operations 
were  disrupted  by  two  failures  of  the  ship’s  only  JP-5 
(F-44)  fuel  pump  that  precluded  refueling  any  embarked 
aircraft  for  long  periods.  In  addition  to  problems  with 
TCDL,  systems  that  support  flight  operations,  such  as 
the  Advanced  Stabilized  Glide  Slope  Indicator,  tactical 
air  navigation  system,  and  the  wind-speed  measurement 
system  were  frequently  degraded  or  inoperative.  These 
failures  had  little  impact  during  the  operational  test 
because  weather  conditions  were  generally  favorable, 
but  in  more  challenging  conditions,  their  failure  could 
severely  limit  flight  operations. 

-  Other  Operations.  COTF  also  exercised  LCS  4  and 
her  crew  in  a  variety  of  other  shipboard  evolutions 
during  OT-C4,  including  anti-terrorism/force  protection, 
damage  control,  mooring  and  unmooring,  refueling  at 
sea,  vertical  replenishment,  man-overboard  recovery, 
communications,  and  receiving  a  tow.  DOT&E 
observers  reported  that  the  ships  performed  as  expected 
during  the  observed  evolutions. 

-  Cybersecurity.  In  the  only  phase  of  operational  testing 
completed  to  date  in  LCS  2,  COTF  conducted  a  CVPAof 
the  seaframe  and  embarked  Increment  1  MCM  mission 
package  in  June  and  July  2015  while  the  ship  was 
moored  in  Pensacola,  Florida,  during  a  comprehensive 
maintenance  availability.  COTF’s  cybersecurity  team 
assessed  all  shipboard  and  mission  package  systems 

that  were  in  scope  except  the  MH-60S  helicopter, 
SeaRAM,  and  software-defined  radios.  The  CVPA 
details  are  classified  but  indicate  that,  like  the  Freedom 
variant  seaframe,  the  Independence  variant  seaframe  has 
cybersecurity  deficiencies  that  significantly  degrade  the 
ship’s  operational  effectiveness.  Plans  for  the  last  phase 
of  the  cybersecurity  operational  testing,  an  Adversarial 
Assessment,  are  on  hold  pending  a  Navy  decision  on  the 
readiness  of  the  Increment  1  MCM  mission  package  and 
Independence  variant  seaframe  for  MCM  operational 
testing.  As  noted  earlier,  all  OT-C4  cybersecurity  testing 
in  LCS  4  has  been  delayed  until  the  Navy  completes 
upgrades  to  the  ship’s  networks  designed  to  enhance  its 
cybersecurity  and  correct  known  issues. 

-  Limitations  on  Watercraft  Launch  and  Recovery. 
Because  of  structural  defects  in  LCS  2  and  LCS  4 
identified  during  rough  water  trials  aboard  LCS  2,  the 
Navy  has  established  a  limit  on  the  maximum  allowable 
dynamic  loading  of  the  Twin-Boom  Extensible  Crane 
(TBEC)  used  to  launch  and  recover  the  RMMV  and 
other  watercraft.  Sea  conditions  that  would  have  caused 
the  limit  to  be  exceeded  precluded  RMS  operations  on 
several  occasions  during  the  MCM  mission  package 
TECHEVAL  aboard  LCS  2.  Additionally,  the  design 

of  the  Independence  variant  seaframe  and  the  ship’s 
watercraft  launch,  handling,  and  recovery  system 
used  with  the  TBEC,  coupled  with  the  turbulent  wake 
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produced  by  the  water  jets,  make  launch  and  recovery  of 
the  RMMV  and  other  watercraft  complex  and  somewhat 
risky  evolutions,  requiring  the  ship’s  crew  to  exercise 
great  care. 

-  Operational  Suitability.  COTF  collected  reliability, 
maintainability,  availability,  and  logistics  supportability 
data  to  support  evaluation  of  the  operational  suitability 
of  the  Independence  variant  seaframe  throughout  the 
last  half  of  FY15  and  plans  to  continue  that  effort  when 
MCM  OT-C2  begins  on  LCS  2  and  when  OT-C4  resumes 
on  LCS  4.  Although  incomplete,  the  data  collected  to 
date  show  that  essential  Independence  variant  seaframe 
systems  have  significant  reliability  problems.  During 
developmental  testing,  the  LCS  4  crew  had  difficulty 

in  keeping  the  ship  operational  as  it  suffered  repeated 
failures  of  the  ship’s  diesel  generators,  water  jets,  and 
air  conditioning  units.  Some  of  the  failures  proved  to 
be  problems  with  communications  between  the  systems 
and  the  Engineering  Control  System,  which  forced  the 
crew  to  place  key  systems  into  ‘local’  mode  to  resume 
operation.  As  a  temporary  expedient,  this  was  generally 
effective,  but  because  the  reduced  size  of  the  crew  was 
predicated  on  extensive  use  of  automation,  the  added 
labor  involved  in  monitoring  and  controlling  these 
systems  individually  stretches  the  limits  of  the  crew’s 
ability  to  operate  and  maintain  the  ship’s  systems.  In 
addition,  because  of  the  planned  reliance  on  shore-based 
contractor  support,  in  many  cases  the  LCS  crew  lacks 
the  documentation,  training,  test  equipment,  and  tools 
required  to  troubleshoot  and  repair  serious  problems 
as  they  emerge.  Lack  of  documentation  and  training 
contributed  to  recurring  issues  with  the  TSCE,  integrated 
combat  management  system  (ICMS)  software,  and 
communications  systems. 

-  LCS  2  Reliability  and  Availability.  LCS  2  equipment 
failures  left  the  ship  with  limited  mission  capability 
throughout  the  176-day  data  collection  period  and  with 
no  mission  capability  on  two  occasions.  Many  of  the 
failures  disrupted  MCM  operations,  and  caused  the  ship 
to  return  to,  or  remain  in,  port  for  repairs.  The  ship 
had  to  call  for  shore-based  assistance  to  repair  nearly 
all  significant  failures.  The  following  are  the  most 
significant  seaframe  equipment  problems  observed 
during  the  data  collection  period. 

■  LCS  2  had  no  Secret  Internet  Protocol  Router 
Network  (S1PRNET)  connectivity  for  a  period  of  four 
days  at  the  beginning  of  the  period  because  of  a  hard 
drive  failure  that  had  occurred  the  previous  month. 
Lack  of  SIPRNET  connectivity  impedes  the  flow 

of  classified  information  between  the  ship  and  the 
operational  commander. 

■  Failure  of  the  navigation  attitude  server  deprived 
critical  combat  systems  of  roll  and  pitch  information 
for  six  days  during  the  period  and  limited  the 
capability  of  ICMS,  SeaRAM,  and  the  AN/SPS-77 
ASR. 


■  SeaRAM  experienced  four  failures,  leaving  the  ship 
with  no  air  defense  capability  for  a  total  of  120  days 
(68  percent  of  the  period). 

■  The  MK  110  57  mm  gun  was  inoperative  for  114  days 
because  of  damage  caused  when  gun  components 
overheated,  rendering  the  ship  incapable  of  any 
defense  against  an  LSF  threat  and  leaving  only 
crew-served  machine  guns  for  defense  against  surface 
threats. 

■  SAFIRE  was  inoperative  for  a  period  of  25  days  until 
the  turret  could  be  replaced,  but  this  outage  occurred 
while  the  57  mm  gun  was  inoperative,  a  period  when 
the  ship  already  had  little  capability  to  defend  against 
a  surface  or  LSF  threat. 

■  The  AN/SPS-77  ASR  had  multiple  outages  of  short 
duration  (3  to  30  minutes)  that  required  the  crew  to 
reboot  an  interface  device  and  was  restricted  to  limited 
use  because  of  a  failing  antenna  turntable  gearbox 

for  a  period  of  3  weeks  until  it  could  be  repaired  by  a 
SAAB  technician. 

■  Failure  of  a  power  conversion  unit  that  supplied 

400  Hertz  power  to  the  mission  bay  deprived  the  ship 
of  MCM  mission  capability  for  20  days  while  the 
ship  was  in  port  undergoing  repairs.  The  Naval  Sea 
Systems  Command  was  forced  to  locate  a  functional 
replacement  because  the  failed  unit  was  obsolete  and 
could  no  longer  be  supported  with  repair  parts. 

■  The  ship  also  lost  the  capability  to  supply  400  Hertz 
power  to  the  aircraft  hangar,  where  it  is  needed  to 
conduct  pre -mission  checks  on  the  MH-60S  and 
AMCM  systems.  The  ship  was  provided  portable 
power  units  to  fill  the  gap  until  the  ship’s  power 
converter  could  be  repaired.  The  Navy  never 
determined  the  cause  of  the  near-simultaneous 
failures  of  the  two  power  conversion  units,  although 
technicians  considered  them  related. 

■  LCS  2  experienced  multiple  air  conditioning 
equipment  failures  and  was  unable  to  supply  enough 
cooling  to  support  the  ship’s  electronics  on  several 
occasions.  One  or  more  of  the  ship’s  3  chilled  water 
units  was  either  inoperative  or  operating  at  reduced 
capacity  for  159  days  (90  percent  of  the  period). 

■  A  Mobicon  straddle  carrier  failure  left  the  ship  unable 
to  conduct  waterborne  MCM  operations  for  a  period 
of  four  days  until  a  technician  could  travel  from 
Australia  to  diagnose  the  problem  and  make  needed 
adjustments.  This  episode  demonstrated  the  crew’s 
paucity  of  documentation,  training,  and  diagnostic 
equipment. 

■  The  boat  davit  failed  while  launching  the  lifeboat 
(7-meter  RHIB)  and  forced  the  ship  to  accompany 
the  boat  into  port.  The  ship  remained  in  port  with 
no  usable  mission  capability  for  five  days  because 
the  lifeboat  is  safety  equipment  and  essential  for 
operations  at  sea. 
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■  The  ship  experienced  several  Ship  Service  Diesel 
Generator  failures  during  the  period,  but  was  never 
without  at  least  two  of  four  generators  operable 
(sufficient  to  power  all  combat  loads,  but  limited 
maximum  propulsion  speed). 

■  LCS  2  was  unable  to  launch  and  recover  RMMVs  on 
15  days  because  of  4  separate  propulsion  equipment 
failures  involving  diesel  engines,  water  jets,  and 
associated  hydraulic  systems  and  piping.  These 
failures  would  also  have  limited  the  ship’s  capability 
to  use  speed  and  maneuver  to  defend  itself  against 
small  boat  threats. 

■  LCS  2  was  unable  to  launch  and  recover  RMMVs  on 
10  additional  days  because  of  3  TBEC  failures. 

LCS  4  Reliability  and  Availability.  LCS  4  exhibited 
equipment  failures  that  limited  its  operational  availability 
and  left  the  ship  with  limited  mission  capability  at 
various  points  throughout  the  data  collection  period 
(113  days).  The  ship  was  fully  mission  capable  less 
than  40  percent  of  that  time.  The  following  are  the 
most  significant  seaframe  equipment  problems  observed 
during  the  data  collection  period. 

■  LCS  4  spent  45  days  during  this  period  without 
all  4  engines  and  steerable  water  jets  operational. 

This  includes  a  19-day  period  in  May  when  3  of 
the  4  engines  were  degraded  or  non-functional. 

Since  LCS  relies  on  speed  to  augment  its  combat 
effectiveness  and  survivability,  the  loss  of  any  engine 
(especially  a  gas  turbine)  can  degrade  the  ship’s 
effectiveness. 

■  LCS  4  experienced  multiple  air  conditioning 
equipment  failures  and  was  unable  to  supply 
enough  cooling  to  support  the  ship’s  electronics 
for  a  two  week  period  in  May.  One  or  more  of  the 
ship’s  3  chilled  water  units  was  either  inoperative  or 
operating  at  reduced  capacity  for  56  days. 

■  JP-5  fuel  pump  failures  left  the  ship  with  no  capability 
to  refuel  the  embarked  helicopter  for  1 1  days. 

■  A  TBEC  failure  left  the  ship  unable  to  recover  an 
11-meter  RH1B  until  the  day  after  it  was  launched. 
Once  the  RH1B  was  recovered,  the  TBEC  remained  in 
a  degraded  state  for  23  days. 

■  The  57  mm  gun  was  either  inoperative  or  operating  in 
a  degraded  condition  for  35  days. 

■  SeaRAM,  the  ship’s  primary  defense  against  ASCMs, 
was  inoperative  or  degraded  for  15  days. 

■  The  ship’s  ride  control  system,  used  for  high-speed 
maneuvering,  did  not  appear  to  be  fully  functional  at 
any  time  during  developmental  or  operational  testing 
in  FY15. 

■  Similar  to  problems  seen  on  LCS  2,  the  AN/SPS-77 
ASR  had  multiple  outages  of  short  duration  (3  to 
30  minutes)  that  required  the  crew  to  reboot  an 
interface  or  the  radar  itself. 

■  Numerous  interruptions  in  the  flow  of  navigation 
data  were  noted  during  live  fire  events  in  September, 


seriously  degrading  the  ship’s  combat  effectiveness. 
Both  combat  and  navigation  systems  require  frequent 
updates  about  the  ship’s  heading,  roll,  and  pitch 
to  operate  correctly.  Without  this  information,  the 
ASR,  SeaRAM,  and  ESM  system  cannot  correctly 
determine  the  relative  orientation  of  targets  to  the 
ship,  and  more  critically,  the  57  mm  gun  cannot  fire. 
Even  a  momentary  interruption  of  navigation  data  to 
these  systems  forces  57  mm  operators  to  reestablish  a 
track  on  the  target  via  SAF1RE  (a  laborious  process) 
and  disrupts  the  crew’s  situational  awareness. 

SUW  Mission  Package 

•  In  FY14  operational  testing,  LCS  3  ( Freedom  variant) 
and  an  embarked  Increment  2  SUW  mission  package 
demonstrated  the  capability  to  defeat  a  small  swarm 
of  FlACs  under  the  specific  conditions  detailed  in  the 
Navy  requirement;  however,  the  crew  received  extensive 
hands-on  training  that  might  not  be  available  to  crews 
on  other  ships.  Testing  conducted  to  date  has  not  been 
sufficient  to  demonstrate  LCS  capabilities  in  more  stressing 
scenarios  consistent  with  existing  threats  or  to  demonstrate 
with  high  confidence  that  the  Freedom  variant  LCS  can 
defeat  even  small  swarms  with  regularity  when  equipped 
with  the  Increment  2  SUW  mission  package. 

•  While  equipped  with  the  Increment  2  SUW  mission 
package,  LCS  4  participated  in  three  engagements  with 
small  swarms  of  FIACs.  The  engagements  used  the 
same  “keep-out”  criteria  as  the  single  target  self-defense 
engagements.  Although  all  of  the  attacking  boats  were 
ultimately  defeated,  an  attacker  managed  to  penetrate 
this  “keep-out”  range  in  two  of  the  three  events.  In  all 
three  events,  however,  the  ship  expended  a  large  quantity 
of  ammunition  from  the  seaframe’s  57  mm  gun  and  the 
two  mission  package  30  mm  guns,  while  contending  with 
repeated  network  communication  faults  that  disrupted  the 
flow  of  navigation  information  to  the  gun  systems  as  well 
as  azimuth  elevation  inhibits  that  disrupted  or  prevented 
establishing  firing  solutions  on  the  targets.  The  SAFIRE 
performance  issues  described  in  the  seaframe  section  also 
presented  the  crew  with  challenges  during  the  swarm 
engagements.  LCS  4’s  failure  to  defeat  this  relatively 
modest  threat  routinely  under  test  conditions  raises 
questions  about  its  ability  to  deal  with  more  realistic  threats 
certain  to  be  present  in  theater. 

•  In  the  past,  the  30  mm  Gun  Mission  Modules  have  been 
prone  to  jams  caused  by  separation  of  ammunition  links 
and  accumulation  of  spent  cartridges  in  the  ejection  path. 
Although  they  can  typically  be  cleared  in  a  few  minutes, 
ammunition  jams  interrupt  firing  and  can  be  sufficiently 
disruptive  to  cause  the  ship  to  lose  valuable  time  in  a 
fast-moving  engagement.  FY14  testing  conducted  in  LCS  3 
showed  the  Navy’s  concerted  effort  to  improve  ammunition 
belts  has  had  some  positive  effect,  but  the  problem  has  not 
been  eliminated.  LCS  4  experienced  numerous  instances 
of  link  separation  during  FY15  developmental  testing,  but 
DOT&E  observers  report  that  modified  ammunition  can  lids 
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introduced  before  the  operational  test  have  largely  mitigated 
that  problem. 

•  LCS  4  experienced  a  large  number  of  azimuth  elevation 
inhibits  during  FY15  developmental  and  operational 
tests,  which  momentarily  interrupted  30  mm  gun  firing 
engagements.  The  azimuth  elevation  inhibit  is  designed  to 
prevent  the  gun  from  firing  when  the  pointing  of  the  gun 
sight  and  gun  are  not  in  reasonable  agreement.  Observers 
reported  that  the  inhibits  occur  with  annoying  frequency 
(a  dozen  or  more  times  during  a  live  fire  engagement), 
severely  impairing  the  flow  of  the  engagement.  The  crew 
reported  that  the  cause  of  the  frequent  inhibits  was  to  have 
been  corrected  in  a  software  patch,  but  the  patch  was  either 
not  installed  or  not  effective. 

MCM  Mission  Package 

•  DOT&E  concluded  in  a  November  2015  memorandum 
to  the  Secretary  of  Defense  and  the  Navy,  based  on  the 
testing  conducted  to  date,  that  an  LCS  employing  the 
current  MCM  mission  package  would  not  be  operationally 
effective  or  suitable  if  it  were  called  upon  to  conduct  MCM 
missions  in  combat  and  that  a  single  LCS  equipped  with 
the  Increment  1  MCM  mission  package  would  provide  little 
or  no  operational  capability  to  complete  MCM  clearance 
missions  to  the  levels  needed  by  operational  commanders. 
The  primary  reasons  for  this  conclusion  are: 

-  Critical  MCM  systems  are  not  reliable. 

-  The  ship  is  not  reliable. 

-  Vulnerabilities  of  the  RMMV  to  mines  and  its  high 
rate  of  failures  do  not  support  sustained  operations  in 
potentially  mined  waters. 

-  RMMV  operational  communications  ranges  are  limited. 

-  Mine  hunting  capabilities  are  limited  in 
other-than-benign  environmental  conditions. 

-  The  fleet  is  not  equipped  to  maintain  the  ship  or  the 
MCM  systems. 

-  The  AMNS  cannot  neutralize  most  of  the  mines  in 
the  Navy’s  threat  scenarios;  an  Explosive  Ordinance 
Disposal  Team  or  other  means  provided  by  another  unit 
must  be  used. 

•  During  the  MCM  mission  package  TECHEVAL,  the  Navy 
demonstrated  that  an  LSC  could  detect,  classify,  identify, 
and  neutralize  only  a  fraction  of  the  mines  in  the  Navy’s 
mine  clearance  scenarios  while  requiring  extraordinary 
efforts  from  shore  support,  maintenance  personnel,  and 
contractors. 

•  During  developmental  testing,  the  Navy  has  not 
demonstrated  that  it  can  sustain  LCS-based  mine 
reconnaissance  and  mine  clearance  rates  necessary 
to  meet  its  strategic  mine  clearance  timelines. 


Following  TECFIEVAL,  DOT&E  identified  seaframe 
reliability  and  availability,  poor  reliability  of  MCM 
components — particularly  the  RMS/RMMV — system 
integration  problems,  and  subsystem  limitations  as  critical 
shortcomings  that  have  substantially  limited  MCM 
effectiveness.  In  addition  to  the  seaframe  problems 
discussed  earlier  in  this  LCS  report,  this  section  discusses 
specific  mission  package  shortcomings  that,  unless 
corrected,  will  continue  to  prevent  the  Navy  from  achieving 
its  LCS  MCM  objectives,  including  the  required  timelines 
for  large-scale  mine  clearance  operations. 

•  As  stated  in  the  November  2015  DOT&E  memorandum 
to  the  Secretary  of  Defense  and  the  Navy,  testing 
continues  to  show  that  employing  these  LCSs  with  the 
Increment  1  MCM  mission  package  would  require  an 
exorbitant  and  costly  shore  infrastructure  to  make  an 
insignificant  contribution  to  the  mine  area  clearance  needs 
of  operational  commanders.  In  the  pre-test  work-ups 
and  the  TECHEVAL,  the  crew  had  to  request  on-site  or 
remote  assistance  33  times.  The  RMMVs  during  this  same 
period  required  291  shore -based  actions  necessitating 
4,123  man-hours  of  effort  to  accomplish  107.7  hours 

of  minehunting.  The  Navy  significantly  increased  the 
shore -based  support  above  their  original  support  concept  to 
complete  the  TECHEVAL. 

•  Inability  to  Sustain  Timely  MCM  Operations.  LCS 

MCM  mission  package  testing  since  2011  has  shown  that 
MCM  mission-critical  systems  are  often  not  available 
when  needed  and  frequently  fail  after  only  short  periods 
of  operation,  making  it  impossible  for  the  Independence 
variant  LCS  to  sustain  timely  MCM  activities  over 
long  periods.  Problems  with  seaframe  support  systems 
(discussed  above),  the  Remote  Minehunting  Module, 
and  MH-60S  and  AMCM  modules  have  all  contributed 
to  lost  MCM  productivity.  During  TECHEVAL,  in 
FY15,  the  Navy  devoted  approximately  80  of  132  test 
days  to  seaframe,  RMS,  and  AMCM  repair  actions 
rather  than  minehunting  operations.  These  TECHEVAL 
corrective  maintenance  demands  prevented  LCS  2  from 
demonstrating  that  it  could  provide  rapid  and  sustained 
mine  reconnaissance  and  mine  clearance. 

-  RMS.  Severe  RMS  reliability  problems  continued 
to  persist  throughout  FY15  testing.  The  table  below 
provides  a  summary  of  RMMV  and  RMS  reliability 
data  collected  that  shows  the  reliability  of  the  RMMV 
and  RMS  are  consistently  below  the  75  hours  Mean 
Time  Between  Operational  Mission  Failure  (MTBOMF) 
prescribed  by  the  Navy  requirements. 
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RMS  and  v6.0  RMMV  Reliability  in  2014-201 5  Testing 


Test  Event 

Test  Period 

System  Operating 
Time  (Hours) 

RMMVOMFs 

RMMV  MTBOMF 
(Hours) 

RMSOMFs 

RMS  MTBOMF 
(Hours) 

LCS  MCM  MP 
DT-B2  Ph4  Pd2 

Sept  11  -  Oct  20, 201 4 

139.0 

3 

46.3 

(20.8-126. 1) 

6 

23.2 

(13.2-44.1) 

DT-B1 

Jan  13 -Mar  25, 2015 

163.4 

7 

23.3 

(13.9-42.0) 

8 

20.4 

(12.6-35.1) 

LCS  MCM  MP 
TECHEVAL 

Apr  7 -Aug  30, 2015 

265.7 

15 

17.7 

(12.5-25.8) 

17 

15.6 

(1 1.3-22.2) 

All 

Sep  1 1, 2014- Aug  30, 2015 

568.1 

25 

22.7 

(17.4-30.1) 

31 

18.3 

(14.4-23.6) 

Note:  Values  in  parentheses  represent  80  percent  confidence  intervals. 

MCM  -  Mine  Countermeasures;  MP  -  mission  package; TECHEVAL  -  Technical  Evaluation;  RMMV  -  Remote  Muti-Mission  Vehicle;  OMF  -  Operational  Mission  Failure; 

MTBOMF  -  Mean  Time  Between  Operational  Mission  Failure 

As  DOT&E  indicated  in  an  August  2015  memorandum 
to  USD(AT&L),  without  changes,  RMMV  and  RMS 
reliability  problems  threaten  the  Navy’s  capacity  to  field 
and  sustain  a  viable  LCS-based  MCM  capability.  Since 
the  RMS  is  critical  to  achieving  the  Navy’s  sustained 
area  coverage  rate  requirement,  this  annual  report  also 
includes  a  separate  article  on  the  RMS  that  provides 
additional  detail. 

■  During  TECHEVAL,  four  RMMVs  and  six 
AN/AQS-20As  operated  off-board  LCS  for  226  hours 
and  conducted  94  hours  of  minehunting  (employing 
the  sonar  to  actively  search  for  mines,  revisit  contacts, 
and  identify  bottom  objects).  On  six  occasions,  an 
RMMV  could  not  be  recovered  aboard  LCS  2  and 
had  to  be  towed  to  port  by  test  support  craft  and  then 
shipped  to  the  remote  operating  site  (simulating  an 
in-theater  depot-level  maintenance  activity)  or  prime 
contractor  site  (original  equipment  manufacturer 
intermediate-  and  depot-level  repair  facility)  for 
repairs.  On  average,  the  LCS  2  completed  a  total  of 

5  hours  of  RMS  minehunting  per  week  (1.25  hours  per 
week  per  RMMV),  and  an  RMMV  had  to  be  towed  to 
port  for  every  16  hours  of  RMS  minehunting. 

■  The  pace  of  RMS  operations  demonstrated  by 

one  LCS  with  4  RMMVs  is  less  than  10  percent  of  the 
operating  tempo  for  a  single  ship  shown  in  the  Navy’s 
Design  Reference  Mission  Profile  for  Increment  1 
bottom-focused  minehunting  (shallow-water) 
operations.  Based  on  the  demonstrated  pace  of 
operations  during  TECE1EVAL,  all  of  the  RMMVs 
the  Navy  plans  to  acquire  to  outfit  24  MCM  mission 
packages  would  be  required  to  search  the  area  that 
the  Navy  originally  projected  a  single  LCS  and  MCM 
mission  package  could  search. 

■  Although  the  Navy  considers  one  of  the  two  RMMVs 
in  the  Increment  1  mission  package  an  embarked 
spare  that  permits  continued  RMS  operations  even 
after  one  unit  fails,  LCS  2  averaged  just  3.5  days 
underway  before  losing  all  RMS  capability,  that 
required  a  call  for  outside  RMS  repair  assistance,  or 
necessitated  a  return  to  port.  LCS  2  was  underway  for 


more  than  one  week  with  at  least  one  mission-capable 
RMS  embarked  only  once  during  TECHEVAL.  On 
five  occasions,  LCS  2  operated  for  less  than  two  days 
before  encountering  an  RMS  problem  that  required 
assistance  from  shore -based  intermediate-level 
maintenance  personnel  to  continue  operations.  In 
three  cases,  an  RMMV  was  recovered  without 
collecting  minehunting  data.  These  problems  resulted 
in  the  RMMV  returning  to  LCS  2  with  at  least  some 
fraction  of  the  expected  mission  data  in  only  15  of 
24  launches  (63  percent). 

■  Mishaps  also  severely  damaged  two  RMMVs,  causing 
them  to  be  returned  to  the  contractor  for  extensive 
repairs. 

■  Despite  underway  periods  that  were  short  relative 
to  the  expectations  of  the  LCS  Design  Reference 
Mission  Profile,  both  RMMVs  embarked  at  the 
beginning  of  an  underway  period  were  unavailable 
to  conduct  minehunting  missions  six  times  during 
TECHEVAL. 

■  On  3  occasions,  totaling  19  days,  all  four  v6.0 
RMMVs  in  the  Navy’s  inventory  were  unavailable  to 
execute  minehunting  missions. 

■  The  Navy  completed  TECHEVAL  with  one  of  four 
RMMVs  operational.  However,  post-test  inspections 
revealed  that  the  sonar  tow  cable  installed  in  that  unit 
was  no  longer  functional. 

-  AMCM.  During  TECHEVAL,  the  MH-60S  and  its 
associated  AMCM  mission  kit  and  mission  systems  also 
experienced  problems  that  interrupted  or  delayed  LCS 
MCM  activities. 

■  Nine  MH-60S  AMCM  problems  interrupted  or 
delayed  MCM  missions.  These  problems  included 
MH-60S  rotor  blade  delamination,  an  MH-60S  power 
distribution  unit  failure,  a  broken  relief  valve  on  an 
MH-60S  hydraulic  reservoir,  multiple  AMCM  mission 
kit  failures  that  required  the  MH-60S  to  return  to  port 
for  repairs,  and  an  AMNS  neutralizer  that  failed  to 
launch  when  commanded.  The  launch  failure  would 
have  required  the  aircrew  to  jettison  the  launch  and 
handling  system  if  live  rounds  (operational  assets) 
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been  employed.  As  a  result,  LCS  2  demonstrated 
sustained  MH-60S  operations  lasting  more  than  one 
week  just  once  during  TECHEVAL. 

■  On  eight  occasions,  LCS  2  conducted  MH-60S 
operations  for  two  days  or  less  before  needing  repairs 
that  in  many  cases  required  the  ship  or  helicopter  to 
return  to  port  for  spare  parts  or  repairs.  In  one  case, 
after  returning  to  port,  the  Navy  elected  to  replace 

a  helicopter  embarked  aboard  LCS  and  in  need  of 
repairs  rather  than  repair  it. 

■  In  total,  during  132  days  of  TECHEVAL,  the  LCS 
2  Aviation  Detachment  employed  two  MH  60S 
helicopters  for  141  flight  hours. 

■  Considering  only  the  58  days  underway,  LCS  2  was 
ALMDS-mission  capable  for  16  days,  AMNS-mission 
capable  for  26  days,  and  not  capable  of  conducting 
the  planned  AMCM  mission  for  16  days  primarily 
because  of  helicopter  and  mission  kit  problems. 

Nearly  all  the  lost  AMCM  mission  days  occurred 

in  the  AMNS  configuration.  This  is  not  surprising 
given  that  the  AMNS  mission  is  more  stressing  on  the 
MH-60S  and  its  AMCM  mission  kit  because  of  the 
need  to  lower  the  loaded  AMNS  launch  and  handing 
system  into  the  water  and  retrieve  it  at  least  once  per 
sortie. 

■  The  MH-60S  aircrew  employed  2  ALMDS  pods  to 
search  for  mines  for  33  hours  and  3  AMNS  launch 
and  handling  systems  to  launch  107  inert  neutralizers 
against  66  targets. 

■  Since  the  MH-60S  AMCM  capability  is  critical 
to  achieving  the  Navy’s  sustained  area  coverage 
rate  requirement,  this  annual  report  also  includes 
a  separate  article  on  the  MH-60S  that  provides 
additional  detail. 

•  Communications  between  LCS  and  its  Unmanned 
Vehicles.  Two  significant  communications  shortcomings 
limit  the  effectiveness  of  the  current  LCS  MCM  mission 
package  system-of-systems.  One  centers  on  the  limited 
range  of  high  data  rate  communications  between  an 
off-board  RMMV  and  the  host  LCS  and  the  other  is  related 
to  the  persistent  difficulty  with  establishing  and  maintaining 
the  existing  line-of-sight  (LOS)  and  over-the-horizon 
(OTH)  communications  channels.  The  former  limits  the 
reach  and  productivity  of  LCS  MCM  operations,  and  the 
latter  results  in  frequent  mission  delays  and  the  potential 
loss  of  an  RMMV  with  which  the  LCS  is  unable  to 
communicate.  Unless  these  problems  are  solved,  the  LCS 
and  its  MCM  mission  package  will  never  be  able  to  fulfill 
its  wartime  MCM  missions  within  the  timelines  required. 

•  Although  the  RMMV  can  search  autonomously  while 
operating  OTH  from  the  LCS,  it  can  only  conduct 
Electro-optical  Identification  operations  to  reacquire 
and  identify  bottom  mines  when  operating  within  LOS 
communications  range  of  the  LCS.  This  limitation 
will  complicate  MCM  operations  in  long  shipping 
channels,  and  will  make  it  necessary  to  clear  a  series  of 


LCS  operating  areas  to  allow  the  ship  to  follow  MCM 
operations  as  they  progress  along  the  channel.  The 
cleared  operating  areas  must  be  close  enough  to  the 
intended  search  area  to  maintain  LOS  communications 
and  large  enough  to  enable  LCS  operations,  including  ship 
maneuver  to  facilitate  launch  and  recovery  of  the  RMMV 
and  MH-60S  helicopter.  The  additional  time  required 
to  clear  these  areas  will  increase  the  demand  for  mine 
clearance.  Although  a  May  2012  Navy  briefing  proposed 
development  of  an  airborne  relay  and  a  high  frequency 
ground  wave  radio  capability,  along  with  other  upgrades, 
to  make  the  Increment  1  MCM  mission  package  “good 
enough”  for  IOT&E,  the  Navy  has  not  yet  fielded  either 
of  those  capabilities.  Had  LCS  2  been  required  to  clear 
its  operating  areas  during  the  2015  TECHEVAL  and  the 
Area  Coverage  rate  Sustained  remained  unchanged,  the 
time  required  to  complete  MCM  operations  in  the  test  field 
would  have  increased  nearly  three-fold.  In  the  May  2012 
briefing  cited  above,  the  Navy  reached  a  similar  conclusion 
regarding  the  operational  consequences  of  limited  RMMV 
communications  ranges. 

•  During  TECHEVAL,  LCS  2  had  frequent  problems 
establishing  initial  communications  between  the  ship  and 
an  RMMV  using  existing  OTH  and  LOS  channels  and 
maintaining  those  communications  links  once  established. 
These  problems  frequently  delayed  the  start  of  RMS 
missions  and  periodically  terminated  missions  prematurely. 
On  one  occasion,  loss  of  communications  during  an  attempt 
to  launch  an  RMMV  caused  the  ship  to  return  to  port  with 
the  RMMV  suspended  from  the  TBEC  because  the  crew 
was  unable  to  complete  the  launch  or  bring  the  vehicle 
back  into  the  mission  bay.  On  another  occasion,  loss  of 
LOS  communications  resulted  in  extensive  damage  to  an 
RMMV  that  required  months  of  depot-level  repair  at  the 
contractor’s  facility  when  the  ship  attempted  to  recover 

it  using  OTH  communications.  On  a  third  occasion,  an 
abrupt  loss  of  power  led  to  loss  of  communications  with 
an  RMMV,  making  it  necessary  for  a  test  support  craft  to 
take  the  RMMV  under  tow.  In  addition  to  these  incidents, 
the  LCS  crew  routinely  found  it  necessary  to  seek  help 
from  shore -based  technicians  to  resolve  communications 
problems.  During  the  latter  portion  of  TECHEVAL,  the 
program  manager  embarked  a  team  of  subject  matter 
experts  to  monitor  LCS  -  RMMV  communications,  assist 
with  troubleshooting,  and  collect  diagnostics.  Shortly 
after  the  TECHEVAL,  the  Program  Office  established  a 
task  force  to  analyze  the  communications  problems  and 
propose  solutions.  The  task  force  has  since  recommended 
a  multi-faceted  approach  that  includes  improving  operating 
and  troubleshooting  documentation  for  the  communications 
system-of-systems,  enhancing  crew  training  in  initialization 
of  communications  links  and  fault  troubleshooting,  and, 
longer  term,  a  reexamination  of  the  communications 
architecture. 

•  Potential  Attrition  of  RMMVs  When  Employed  in 
Mined  Waters.  The  combination  of  acoustic  radiated 
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noise,  frequent  RMMV  failures  that  prevent  recovery 
aboard  LCS,  and  the  probability  the  vehicle  and  its  sensor 
will  get  entangled  with  mines  or  other  hazards  all  pose 
a  risk  to  losing  the  RMS.  Given  the  limited  existing 
inventory  of  RMMVs  (four  v6.0  vehicles,  four  vehicles 
awaiting  upgrades  to  v6.0,  and  two  vehicles  designated  for 
training  use  only),  any  RMMV  attrition  would  severely 
degrade  the  Navy’s  ability  to  conduct  LCS-based  MCM 
operations. 

-  RMMV  acoustic  radiated  noise  measurements,  last 
collected  during  developmental  testing  in  2007/2008, 
indicated  that  existing  RMMVs  might  be  vulnerable  to 
some  mines.  The  RMS  Program  Office  has  not  assessed 
radiated  noise  following  recent  vehicle  configuration 
changes  and  has  requested  a  waiver  to  deploy  the  system 
even  through  it  did  not  previously  meet  its  acoustic 
radiated  noise  specification.  If  RMMV  radiated  noise 
continues  to  exceed  acceptable  limits,  systems  could  be 
lost  during  LCS-based  minehunting  and  mine  clearance 
operations  depleting  the  Navy’s  limited  inventory  of 
assets.  The  magnetic  signature  of  the  v6.0  RMMV  has 
not  been  measured. 

-  As  noted  earlier,  only  18  of  24  RMMVs  launched  from 
LCS  2  ended  with  an  RMMV  recovery  aboard  LCS  2 
during  TECHEVAL.  Frequent  RMMV  failures  that 
preclude  vehicle  recovery  aboard  LCS  might  result 

in  lost  RMMVs  and  expose  personnel  who  attempt  to 
recover  RMMVs  in  open  waters  to  air,  surface,  and 
mine  threats.  Because  of  the  number  of  incidents  in 
which  an  RMMV  could  not  be  recovered,  the  Navy 
is  now  considering  options  that  would  provide  LCS 
with  additional  support  to  recover  RMMVs  that  it 
cannot  recover  otherwise.  On  four  occasions  during 
TECHEVAL,  RMMV  failures  precluded  LCS  2  from 
controlling  the  movements  of  an  off-board  RMMV.  If 
similar  failures  occur  during  operations,  the  RMMV 
could  become  disabled  in  the  minefield  or  drift  into 
a  minefield  before  salvage  or  support  craft  arrive  to 
recover  it. 

-  Even  though  test  minefields  are  deliberately  planned 
to  reduce  the  risk  of  RMS  striking  a  mine  target  or 
becoming  entangled  in  its  mooring  cable,  the  RMS 
has  snagged  several  tethered  mines,  and  other  surface 
and  underwater  objects  during  testing.  These  incidents 
often  cause  damage  to  the  vehicle  or  its  deployed  sonar 
that  leaves  the  system  inoperable.  In  some  cases, 
divers  embarked  on  test  support  craft  have  entered 

the  water  to  assist  in  recovery  of  assets  following  a 
snag.  Although  the  Navy  is  still  developing  CONOPS 
to  handle  these  situations  during  operations  in  a  threat 
minefield,  it  is  clear  that  if  these  incidents  occur  during 
wartime  operations  they  will  pose  a  risk  to  vehicles  and 
potential  recovery  personnel.  Furthermore,  the  repeated 
occurrence  of  these  incidents  presents  both  a  tactical 
and  a  system  design  challenge  for  the  Navy  to  resolve 


as  it  tries  to  minimize  attrition  when  the  system  is 
operationally  employed. 

-  In  FY15,  the  Navy  also  disclosed  that  the  AN/AQS-20 
does  not  trail  directly  behind  the  RMMV  when  deployed 
to  tactical  minehunting  depths.  Instead,  the  sensor  tows 
to  starboard  of  the  RMMV  path.  This  offset  causes 

the  RMS  to  behave  like  a  mine  sweeping  system  as  the 
sonar  and  its  tow  cable  passes  through  the  water,  thereby 
increasing  the  risk  of  snagging  a  tethered  mine. 

•  System  Minehunting  Performance  in  Less  Than 
Optimal  Conditions.  Testing  has  revealed  several 
shortcomings  that,  unless  corrected,  will  delay  completion 
of  LCS-based  mine  reconnaissance  and  mine  clearance 
operations. 

-  The  ALMDS  does  not  meet  Navy  detection/classification 
requirements  in  all  depth  bins  or  the  Navy’s  requirement 
for  the  average  probability  of  detection  and  classification 
in  all  conditions  over  a  region  of  the  water  column 

that  extends  from  the  surface  to  a  reduced  maximum 
depth  requirement.  When  the  system  and  operator 
detect  and  classify  a  smaller  percentage  of  mines  than 
predicted  by  fleet  planning  tools,  the  MCM  commander 
will  likely  underestimate  the  residual  risk  to  transiting 
ships  following  clearance  operations.  To  account  for 
this  uncertainty,  the  Navy  might  find  it  necessary  to 
conduct  minesweeping  operations.  However,  the  Navy 
does  not  plan  to  include  the  mechanical  minesweeping 
capability  that  would  be  required  in  the  MCM  mission 
package.  In  some  conditions,  the  ALMDS  also  generates 
a  large  number  of  false  classifications  (erroneous 
indications  of  mine-like  objects)  that  can  delay 
near-surface  minehunting  operations  until  conditions 
improve  or  slow  mine  clearance  efforts  because  of  the 
need  for  additional  search  passes  to  reduce  the  number 
of  false  classifications.  In  favorable  environmental 
conditions,  the  Navy’s  new  multi-pass  tactic  has  been 
successful  in  reducing  false  classifications  to  the  Navy’s 
acceptable  limits  at  the  cost  of  requiring  more  search  and 
identification  time. 

-  The  RMS  program  has  not  yet  demonstrated  that  the 
AN/AQS-20A  operating  in  its  tactical  single  pass  modes 
can  meet  its  detection  and  classification  requirements 
against  deep  water  targets  moored  near  the  ocean  bottom, 
near-surface  moored  mines  that  are  not  detected  by  the 
ALMDS,  or  stealthy  bottom  mines.  Unless  corrected, 
these  problems  will  likely  adversely  affect  the  quality  of 
LCS-based  minehunting  and  mine  clearance  operations 
in  some  threat  scenarios.  As  an  alternative,  additional 
RMS  search  passes  could  be  employed  with  the  sensor  at 
other  depths,  but  this  will  further  slow  minehunting  and 
mine  clearance  operations. 

-  The  results  of  developmental  and  integrated  testing  to 
date  continue  to  show  that  the  RMS’s  AN/AQS-20A 
sensor  does  not  meet  Navy  requirements  for  contact 
depth  localization  accuracy  or  false  classification 
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density  (number  of  contacts  erroneously  classified  as 
mine-like  objects  per  unit  area  searched).  Contact 
depth  localization  problems  complicate  efforts  to 
complete  identification  and  neutralization  of  mines. 

False  classifications,  unless  eliminated  from  the  contact 
list,  require  identification  and  neutralization  effort, 
result  in  the  expenditure  of  limited  neutralizer  assets, 
and  negatively  affect  the  LCS  sustained  area  coverage 
rate.  To  mitigate  the  problem  of  false  classifications, 
the  Navy  has  implemented  tactics  and  software 
designed  to  compare  the  results  of  multiple  search 
passes  over  the  same  area  to  “prune  out”  most  false 
classifications  and  minimize  the  number  conveyed  for 
identification/ neutralization.  Under  some  conditions,  the 
Navy  has  demonstrated  these  pruning  tactics  reduce  false 
classification  densities  to  the  Navy’s  acceptable  limits. 
However,  as  observed  during  developmental  testing 
in  1QFY15,  these  new  procedures  do  not  reduce  false 
classification  densities  appreciably  in  all  operationally 
relevant  conditions.  The  continued  need  for  additional 
passes  to  “prune  out”  excessive  classifications  will 
prevent  the  LCS  MCM  mission  package  from  achieving 
the  Navy’s  predictions  for  Sustained  Area  Coverage 
Rates  that  were  based  on  the  expectation  that  RMS 
would  be  a  “single-pass”  system. 

-  The  Navy  is  developing  AN/AQS-20  pre-planned 
product  improvements  (P3I)  as  a  longer-term  solution  to 
improve  probability  of  correct  classification,  reduce  false 
classifications,  and  resolve  contact  localization  accuracy 
problems,  in  early  FY15,  the  Navy  was  optimistic  that 

it  could  produce  a  mature  P3I  system  prior  to  the  first 
phase  of  LCS  MCM  operational  testing  then  planned 
in  late  FY15.  The  Program  Office  now  expects  the  P3I 
system  to  enter  operational  testing  in  FY18. 

-  Developmental  testing  of  the  RMS  in  2008  revealed  that 
the  system  had  problems  reacquiring  bottom  objects 

for  identification  in  deeper  waters.  Although  the  Navy 
implemented  fixes  in  the  v6.0  RMMV  designed  to 
correct  this  deficiency,  the  Navy  has  not  yet  conducted 
sufficient  testing  to  evaluate  the  efficacy  of  its  fix. 

-  During  an  AN/AQS-20A  operational  assessment  in  2012, 
operators  had  difficulty  identifying  bottom  objects  in 
areas  with  degraded,  but  operationally  relevant,  water 
clarity.  Unless  system  performance  in  this  environment 
improves,  degraded  water  clarity  will  delay  MCM 
operations. 

•  Limited  Mission  Package  Neutralization  Capability.  The 

current  increment  of  the  MCM  mission  package  cannot 
neutralize  moored  mines  above  the  AMNS  operating  ceiling; 
an  Explosive  Ordinance  Disposal  Team  or  other  means 
provided  by  another  unit  must  be  used.  Unfortunately, 
this  limitation  will  preclude  neutralizing  most  of  the  mines 
expected  in  some  likely  threat  scenarios.  Within  its  operating 
range,  AMNS  performance  is  frequently  degraded  by  the  loss 
of  fiber-optic  communications  between  the  aircraft  and  the 
neutralizer.  The  system  has  experienced  loss  of  fiber-optic 


communications  in  a  wide  range  of  operationally  relevant 
operating  conditions,  including  those  that  are  relatively  benign. 
Although  the  Program  Office  has  stated  that  it  intends  to 
develop  an  improved  AMNS  to  extend  its  depth  range  and 
potentially  improve  performance  in  coarse  bottom  conditions 
and  higher  currents,  none  of  these  efforts  are  funded.  The 
Navy  is  also  considering  other  alternatives. 

-  AMNS  Increment  1  cannot  neutralize  near  surface 
mines  because  of  safety  interlocks  designed  to  protect 
the  helicopter  and  crew  from  exposure  to  fragments, 
surge,  and  blast  that  might  result  from  mine  detonation; 
an  Explosive  Ordinance  Disposal  Team  or  other  means 
provided  by  another  unit  must  be  used. 

-  During  the  shore-based  phase  of  an  operational 
assessment  completed  in  2014,  the  system  and  its 
operators  were  unable  to  achieve  the  Navy’s  requirement 
for  mine  neutralization  success  in  realistic  conditions. 
Frequent  loss  of  fiber-optic  communications  between 
the  aircraft  and  the  neutralizer  was  the  primary  cause 

of  unsuccessful  attack  runs.  The  Navy  attributed  the 
failures  to  the  bottom  composition  even  though  the 
bottom  conditions  experienced  in  the  test  area  were 
not  significantly  different  from  those  expected  in  some 
potential  operating  areas. 

-  Following  developmental  testing  in  high-current 
environments  in  2013,  Navy  Air  Test  and  Evaluation 
Squadron  Twenty  One  (HX-21)  concluded  that  the 
AMNS  destructor,  as  currently  designed,  is  ineffective 
in  swift  water  currents.  Although  the  Navy  completed 
additional  developmental  testing  in  2015,  the  Navy’s 
testing  has  not  characterized  system  performance  under 
operationally  realistic  conditions  in  even  moderate 
currents  that  might  be  encountered  in  potential  operating 
areas. 

-  Inability  to  Maintain  Systems.  An  earlier  section  of 
this  LCS  report  noted  that,  consistent  with  the  CONOPS, 
the  LCS  is  reliant  on  shore -based  support  for  assistance 
with  diagnosis  and  repair  of  seaframe  equipment 
problems  and  that  the  ship  could  be  more  self-reliant 

if  the  sailors  were  provided  with  better  maintenance 
training,  technical  documentation,  test  equipment,  and 
tools  and  a  more  extensive  stock  of  spares.  This  holds 
true  for  the  MCM  mission  systems  as  well,  because 
the  mission  package  detachment  is  also  not  equipped 
to  handle  anything  beyond  relatively  uncomplicated 
preventive  maintenance  and  minor  repairs.  For  example, 
the  Navy’s  records  show  that  shore-based  RMMV 
maintenance  personnel  completed  more  than  4,000  hours 
of  RMMV  maintenance  over  6  months  of  TECHEVAL 
work-ups  and  testing  to  support  approximately  108  hours 
of  RMS  minehunting.  Not  only  is  this  level  of  support, 
38  hours  of  maintenance  per  hour  of  minehunting,  far 
beyond  the  capability  of  the  embarked  crew,  it  is  also 
not  sustainable  for  wide-area  LCS  MCM  operations  that 
must  be  completed  quickly. 
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•  Problems  with  Developmental  MCM  Systems. 

Two  problems  observed  during  early  developmental  testing 
of  COBRA  Block  I,  if  not  subsequently  corrected,  could 
adversely  affect  the  operational  effectiveness  and  suitability 
of  the  system  and  the  Increment  2  MCM  mission  package. 

-  During  early  developmental  testing  of  the  COBRA 
Airborne  Payload  System  (CAPS)  on  a  UH-1  helicopter, 
the  system  suffered  multiple  power  losses  because  of  an 
unstable  power  supply  voltage  to  the  power  distribution 
assembly  (PDA)  caused  by  a  bad  reference  ground.  The 
PDA  subsequently  shut  down  CAPS  as  a  precautionary 
measure,  resulting  in  the  loss  of  imagery. 

-  During  dynamic  conditions,  such  as  roll  and  pitch 
maneuvers,  the  COBRA  Integrated  Gimbal  (IG)  was 
unable  to  maintain  the  correct  step-stare  sequence  to 
acquire  a  complete  dataset.  During  flight  operations,  the 
IG  must  continuously  look  at  a  single  spot  (stare)  while 
the  system  records  multiple  images.  The  IG  must  also 
adjust  its  look  angle  to  step  to  the  next  spot  to  optimize 
its  imagery  acquisition.  The  inability  to  maintain  the 
correct  step-stare  sequence  can  result  in  gaps  in  the 
imagery  of  the  target  area. 

ASW  Mission  Package 

•  Although  the  Navy  did  not  conduct  any  ASW  mission 
package  testing  in  FY15,  problems  observed  in  early 
developmental  testing,  if  not  corrected,  could  adversely 
affect  the  operational  effectiveness  and  suitability  of 
the  mission  package  during  a  future  operational  test.  In 
particular,  the  mission  package  exceeds  the  LCS  mission 
package  weight  allowance.  The  weight  of  the  Variable 
Depth  Sonar  and  its  handling  system  is  a  major  contributor, 
and  the  Navy  is  pursuing  weight  reduction  initiatives. 

LFT&E 

•  Neither  LCS  variant  is  expected  to  be  survivable  in 
high-intensity  combat  because  the  design  requirements 
accept  the  risk  that  the  ship  must  be  abandoned  under 
circumstances  that  would  not  require  such  an  action  on 
other  surface  combatants.  Although  the  ships  incorporate 
capabilities  to  reduce  their  susceptibility  to  attack,  previous 
testing  of  analogous  capabilities  in  other  ship  classes 
demonstrates  it  cannot  be  assumed  LCS  will  not  be  hit 

in  high-intensity  combat.  As  designed,  the  LCS  lack  the 
redundancy  and  the  vertical  and  longitudinal  separation  of 
equipment  found  in  other  combatants.  Such  features  are 
required  to  reduce  the  likelihood  that  a  single  hit  will  result 
in  loss  of  propulsion,  combat  capability,  and  the  ability  to 
control  damage  and  restore  system  operation. 

•  LCS  does  not  have  the  survivability  features  commensurate 
with  those  inherent  in  the  USS  Oliver  Hazard  Perry  class 
Guided  Missile  Frigate  (FFG  7)  it  is  intended  to  replace. 

The  FFG  7  was  designed  to  retain  critical  mission  capability 
and  continue  fighting  if  need  be  after  receiving  a  significant 
hit. 

•  The  LCS  3  TSST  revealed  significant  deficiencies  in  the 
Freedom  variant  design.  Much  of  the  ship’s  mission 
capability  would  have  been  lost  because  of  damage  caused 


by  the  initial  weapons  effects  or  from  the  ensuing  fire. 

The  weapons  effects  and  fire  damage  happened  before  the 
crew  could  respond,  and  the  ship  does  not  have  sufficient 
redundancy  to  recover  the  lost  capability.  Some  changes 
could  be  made  to  make  the  ship  less  vulnerable  and 
more  recoverable  without  major  structural  modifications. 
Examples  include  providing  separation  for  the  water  jet 
hydraulic  power  units,  redesigning  the  Machinery  Plant 
Control  and  Monitoring  System,  and  reconfiguring  the 
chilled  water  system  into  a  zonal  system  with  separation  for 
the  air  conditioning  (chilled  water)  plants. 

•  DOT&E  is  analyzing  the  initial  internal  blast  test  findings 
recently  provided  by  the  Navy.  The  Navy  delayed 
completion  of  the  planned  fire  testing  and  final  internal  blast 
tests  until  the  spring  of  2016  because  of  other  Navy  testing 
priorities. 

Recommendations 

•  Status  of  Previous  Recommendations. 

-  The  Navy  partially  addressed  one  FY09  recommendation 
to  develop  an  LFT&E  program  with  the  approval  of  the 
LFT&E  Management  Plan;  however,  the  lethality  testing 
of  the  new  surface-to-surface  missile  still  needs  to  be 
developed. 

-  The  Navy  partially  addressed  the  FY10  recommendations 
to  implement  recommendations  from  DOT&E ’s  Combined 
Operational  and  Live  Fire  Early  Fielding  Report  and  plans 
to  address  other  recommendations  in  future  ships. 

-  With  respect  to  FY 1 1  recommendations  regarding 
AN/AQS-20A  and  ALMDS,  the  Navy  is  adjusting  tactics 
and,  for  the  AN/AQS-20A,  funding  improvements  to 
address  deficiencies.  The  FY11  recommendation  for  the 
Navy  to  continue  to  report  vulnerabilities  during  live  fire 
tests  remains  valid. 

-  For  FY12  recommendations: 

■  The  Navy  partially  addressed  the  recommendations  to 
complete  the  revised  capabilities  document  defining  the 
incremental  approach  to  fielding  mission  packages. 

■  The  Navy  has  released  requirements  letters  for 
Increments  1  and  2  SUW  and  Increment  1  MCM  mission 
packages  only;  however,  the  requirements  have  not  been 
codified  in  approved  Capabilities  Production  Documents. 
The  Navy  published  the  LCS  Platform  Wholeness 
Concept  of  Operations  Revision  D  in  January  2013. 

■  The  Navy  has  not  published  the  concept  of  employment 
for  all  the  mission  packages,  but  advises  that  it  has 
completed  initial  manning  level  studies.  The  Navy  has 
adjusted  ship  and  mission  package  manning  levels  and  is 
continuing  studies  to  determine  the  final  manning  levels. 

■  The  Navy  has  stated  that  gun  reliability  problems 
identified  during  the  Quick  Reaction  Assessment 
conducted  aboard  LCS  1  have  been  resolved  based  on 
limited  testing  conducted  in  October  2012.  Subsequent 
testing  has  demonstrated  that  the  gun  reliability  has 
indeed  improved. 
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■  The  Navy  conducted  LCS-based  phases  of  the  planned 
operational  assessments  of  the  MH-60S  Block  2/3  and 
ALMDS  and  the  MH-60S  Block  2/3  and  AMNS  MCM 
systems  in  1QFY15. 

■  Throughout  FY13/14,  the  Navy  focused  on  correction  of 
material  deficiencies  with  seaframe  launch  and  recovery 
systems,  and  procedural  and  training  deficiencies  that 
prevented  safe  shipboard  launch  and  recovery  of  the 
RMS.  Although  the  Navy  has  retired  some  problems, 
LCS  2  continued  to  experience  some  damage  to 
equipment  during  RMMV  launch  and  recovery  in  low  to 
moderate  sea  states. 

■  The  Navy  should  still  address  the  FY13  recommendation 
to  provide  a  surface-to-surface  missile  LFT&E 
Management  Plan  for  DOT&E  approval  for  the  recently 
selected  surface-to-surface  missile. 

-  For  FY14  recommendations: 

■  The  Navy  continues  to  monitor  the  reliability  of 
LCS  systems  and,  when  warranted  by  available  data, 
incorporates  system  changes  to  improve  reliability  and 
other  aspects  of  performance  as  funding  permits. 

■  The  Navy  has  planned  corrective  actions  for  the 
cybersecurity  deficiencies  identified  during  operational 
testing  of  the  Freedom  and  Independence  variants 

of  LCS  but  installation  of  upgrades  will  be  done  in 
FY16.  The  Navy  completed  a  CVPA  in  LCS  2  with 
the  MCM  mission  package  in  FY15,  but  the  schedule 
for  the  follow-on  Adversarial  Assessment  has  not  been 
determined.  The  Navy  should  consider  scheduling  the 
Adversarial  Assessment  after  the  planned  upgrade  to  the 
ship’s  cybersecurity  configuration  as  was  done  for  the 
LCS  4  with  the  SUW  mission  package,  whose  testing 
will  now  be  done  in  2QFY16  when  it  expects  to  complete 
its  first  phase  of  cybersecurity  upgrades. 

■  The  Navy  has  not  yet  altered  its  plan  for  live  fire  swarm 
engagements  during  testing  of  the  SUW  mission  package; 
testing  conducted  in  LCS  4  duplicated  that  completed  in 
LCS  3  in  FY14.  Nor  has  the  Navy  developed  plans  for 
testing  Increments  3  and  4  of  the  SUW  mission  package. 

■  Although  the  Navy  has  identified  potential  solutions, 
DOT&E  is  not  aware  of  any  funded  effort  to  provide  the 
OTH  communication  needed  for  RMS  electro-optical 
identification  operations. 

■  Although  the  Navy  is  continually  working  to  improve 
mission  system  (RMMV,  ALMDS,  AMNS,  AMCM 
mission  kit,  AN/AQS-20A)  reliability,  FY15  testing 
showed  that  reliability,  maintainability,  and  availability 
problems  continue  to  prevent  timely  and  sustained  MCM 
operations  and  require  extensive  reliance  on  shore -based 
support. 

■  The  Navy  made  minor  modifications  to  the  AMNS 
system  and  trained  operators  to  maintain  forward 
neutralizer  motion  to  reduce  the  risk  of  cutting 
the  fiber-optic  cable,  but  the  system  continued  to 
have  problems  with  early  termination  of  fiber-optic 
communications  during  TECHEVAL.  The  Navy  should 


continue  to  monitor  AMNS  operations  to  identify 
uncorrected  causes  of  fiber  breaks. 

■  The  Navy  reported  that  a  technical  group  is  reviewing  the 
ventilation  lineup  during  condition  ZEBRA,  (the  highest 
condition  of  material  readiness)  in  the  Freedom  variant 
LCS  to  determine  if  the  system  is  operating  as  intended. 

■  The  Program  Office  reports  that  the  contractor  is 
investigating  problems  with  the  Machinery  Plant  Control 
and  Monitoring  System  fire  alarm  system  in  the  Freedom 
variant  LCS. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Shift  to  a  performance -based  test  schedule  rather  than 
continuing  a  schedule-driven  program  to  provide  the  LCS 
program  ample  time  and  resources  needed  to  correct  the 
numerous  serious  problems  that  repeatedly  have  been 
identified  before  operational  testing  occurs. 

2.  Accelerate  efforts  to  obtain  the  intellectual  property  rights 
needed  to  develop  high-fidelity  digital  models  of  the 
AN/SPS-75  and  AN/SPS-77  radars  for  the  PRA  Test  Bed, 
or  present  plans  to  enhance  air  warfare  testing  aboard  the 
self-defense  ship  for  DOT&E  to  review. 

3.  Improve  the  shock  resistance  of  mission-critical  electronics 
in  the  Independence  variant  LCS  to  improve  continuity 

of  operations  during  57  mm  gun  engagements  and  other 
shock-inducing  activities/events. 

4.  Work  with  the  vendor  to  develop  SAFIRE  changes  needed 
to  improve  the  human-machine  interface,  reduce  the  time 
required  to  develop  a  new  track,  improve  tracking,  and 
correct  other  performance  issues  noted  in  FY15  testing 

in  order  to  enhance  the  Independence  variant  seaframe ’s 
effectiveness  against  surface  and  LSF  threats. 

5.  Investigate  and  correct  the  causes  of  Independence  variant 
seaframe  problems  that  disrupt  gunnery  engagements  and 
other  operations,  including  loss  of  navigation  information  to 
combat  systems,  30  mm  gun  azimuth-elevation  inhibits,  and 
the  57  mm  gun’s  azimuth-dependent  range  errors. 

6.  Re-engineer  the  Multi- Vehicle  Communication  System, 
RMMV,  and/or  other  essential  system-of-systems 
components  to  improve  interoperability  and  enable  reliable 
LOS  and  OTH  communications  between  LCS  and  RMMVs. 

7.  Develop  a  safe  method  to  realistically  test  the  ships’  ability 
to  counter  LSF  threats. 

8.  Provide  LCS  crews  with  better  training,  technical 
documentation,  test  equipment,  and  tools,  along  with 
additional  spares  to  improve  the  crews’  self-sufficiency  and 
enhance  LCS  and  mission  package  maintainability. 

9.  Acquire  additional  organic  U.S.  Navy  expertise  in  LCS 
systems  to  reduce  the  reliance  on  equipment  vendors  and 
other  contractors,  particularly  those  located  overseas. 

10.  Continue  to  investigate  options  to  re-engineer  the  recovery 
of  watercraft  in  order  to  reduce  risk,  delays,  crew  workload, 
and  the  likelihood  of  failures. 

11.  Develop  tactics  to  mitigate  system  vulnerabilities  to  mines, 
mine  collision,  and  entanglement  hazards,  and  other  surface 
and  underwater  hazards. 
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MH-60R  Multi-Mission  Helicopter 


Executive  Summary 

•  Commander,  Operational  Test  and  Evaluation  Force  (COTF) 
completed  a  Quick  Reaction  Assessment  (QRA)  of  the 
LAU-61G/A  2.75-inch  Digital  Rocket  Launcher  (DRL) 
armed  with  the  Advanced  Precision  Kill  Weapons  System  II 
(APKWS  II)  rockets  in  1QFY15  in  response  to  a  U.S.  Central 
Command  Urgent  Operational  Need  for  enhanced  Surface 
Warfare  capability  against  threat-representative  small-boat 
targets. 

•  On  May  21,  2015,  DOT&E  provided  the  Secretary  of  the 
Navy  a  classified  Early  Fielding  Summary  and  Scripted  Brief 
that  assesses  the  QRA  test  results. 

•  The  MH-60R  equipped  with  the  DRL  and  APKWS  II  rockets 
demonstrated  marginal  Surface  Warfare  capability  based  on 
the  limited  QRA  testing. 

System 

•  The  MH-60R  is  a  ship-based  medium  lift  helicopter  designed 
to  replace  two  different  helicopters,  the  SH-60B  and  SH60F, 
and  operate  from  cruisers,  destroyers,  frigates,  littoral  combat 
ships,  and  aircraft  carriers. 

•  It  incorporates  dipping  sonar  and  sonobuoy  acoustic  sensors, 
multi-mode  radar,  electronic  warfare  sensors,  a  forward 
looking  infrared  sensor  with  laser  designator,  and  an  advanced 
mission  data  processing  system. 

•  It  employs  MK  46  and  MK  54  torpedoes,  HELLFIRE 
air-to-ground  missiles,  2.75-inch  family  of  rockets,  and  crew 
served  mounted  machine  guns. 

•  It  has  a  three-man  crew:  two  pilots  and  one  sensor  operator. 


Mission 

The  Maritime  Component  Commander  employs  the  MH-60R 

from  ships  or  shore  stations  to  accomplish  the  following: 

•  Surface  Warfare,  Undersea  Warfare,  Area  Surveillance, 
Combat  Identification,  and  Naval  Surface  Fire  Support 
missions  previously  provided  by  the  SH-60B  and  SH-60F 

•  Support  missions  such  as  Search  and  Rescue  at  sea  and,  when 
outfitted  with  necessary  armament,  maritime  force  protection 
duties 

Major  Contractors 

•  Sikorsky  Aircraft  Corporation  -  Stratford,  Connecticut 

•  Lockheed  Martin  Mission  System  and  Sensors  -  Owego, 

New  York 


Activity 

•  COTF  conducted  a  QRA  of  the  MH-60R  equipped  with  the 
DRL  and  APKWS  II  rockets  in  1QFY15  in  response  to  a 
U.S.  Central  Command  Urgent  Operational  Need  for  enhanced 
Surface  Warfare  capability  against  threat-representative 
small-boat  targets.  This  QRA  provided  limited  testing  of 
MH-60R  capability  for  Surface  Warfare  when  equipped  with 
the  APKWS  II  rocket.  Testing  was  conducted  in  accordance 
with  a  DOT&E-approved  test  plan. 

•  DOT&E  provided  the  Secretary  of  the  Navy  a  classified  Early 
Fielding  Summary  and  Scripted  Brief  on  May  21,  2015,  that 
assesses  the  QRA  test  results. 

•  The  Navy  is  preparing  a  Test  and  Evaluation  Master  Plan 
update  for  MH-60R  FOT&E. 

•  The  MH-60R  FOT&E  is  not  currently  scheduled. 

Assessment 

•  The  MH-60R  equipped  with  the  DRL  and  APKWS  II  rockets 
demonstrated  marginal  Surface  Warfare  capability  based  on 
QRA  testing. 


•  The  limited  scope  of  the  QRA  was  not  sufficient  to  fully 
characterize  the  mission  effectiveness  of  the  MH-60R 
equipped  with  the  DRL  and  APKWS  II  rockets  against 
threat-representative  small-boat  targets.  The  Navy  needs  to 
plan  to  conduct  a  complete  operational  test  as  soon  as  possible. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  partially 
addressed  the  FY12  recommendation  to  test  corrections 
made  to  resolve  identified  Multi-Spectral  Targeting  System 
(MTS)  deficiencies  in  conducting  FOT&E.  The  Navy  has 
not  acted  or  has  yet  to  complete  action  on  FY13  and  FY14 
recommendations  to: 

1 .  Conduct  comprehensive  live  fire  lethality  testing 
of  the  HELLFIRE  missile  against  a  complete  set  of 
threat-representative  small-boat  targets  and  to  test  the 
Surface  Warfare  mission  capability  of  MH-60R  equipped 
with  HELLFIRE  missiles. 
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2.  Correct  the  remaining  deficiencies  with  the  MTS  tracker. 

3.  Demonstrate  the  Surface  Warfare  mission  capability  of  the 
MH-60R  helicopter  equipped  with  the  HELLFIRE  missile 
and  MTS  throughout  the  operational  mission  environment 
in  FOT&E  and  LFT&E. 

4.  Conduct  further  evaluation  of  the  Automatic  Radar 
Periscope  Detection  and  Discrimination  (ARPDD) 
employment  in  high-clutter,  high-contact  density  littoral 
environments. 

5.  Address  the  additional  six  recommendations  in  the 
classified  IOT&E  report  on  MH-60R  with  the  ARPDD 
system. 


6.  Investigate  and  correct  interoperability  deficiencies  of  the 
MK  54  lightweight  torpedo  with  MF1-60R  weapons  control 
systems. 

•  FY15  Recommendation. 

1.  The  Navy  should  conduct  MF1-60R  FOT&E  to  test  the  DRL 
armed  with  the  APKWS  II  rocket  and  fully  characterize 
Surface  Warfare  mission  capability  and  lethality  of  this 
capability  against  the  small-boat  threat  targets. 
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MH-60S  Multi-Mission  Combat  Support  Helicopter 


Executive  Summary 

•  DOT&E  concluded  in  a  November  2015  memorandum  to  the 
USD(AT&L)  and  the  Navy,  based  on  the  testing  conducted  to 
date,  that  a  Littoral  Combat  Ship  (LCS)  employing  the  current 
Mine  Countermeasures  (MCM)  mission  package  would  not 
be  operationally  effective  or  operationally  suitable  if  the  Navy 
called  upon  it  to  conduct  MCM  missions  in  combat.  Three  of 
the  seven  primary  shortcomings  supporting  this  conclusion  are 
attributed,  at  least  in  part,  to  the  MH-60S  with  Airborne  Mine 
Countermeasures  (AMCM)  systems: 

-  Critical  MCM  systems  are  not  reliable. 

-  Minehunting  capabilities  are  limited  in  other-than-benign 
environmental  conditions. 

-  The  Airborne  Mine  Neutralization  System  (AMNS),  by 
design,  cannot  neutralize  most  of  the  mines  in  the  Navy’s 
threat  scenarios;  an  Explosive  Ordinance  Disposal  Team 
provided  by  another  unit  must  be  used. 

•  From  April  through  August  2015,  the  Navy  conducted 
Technical  Evaluation  (TECHEVAL)  of  the  Independence 
variant  LCS  and  Increment  1  MCM  mission  package, 
including  the  MH-60S  and  its  AMCM  systems,  aboard  LCS 
2.  During  TECHEVAL,  the  MH-60S  and  its  associated 
AMCM  mission  kit  and  mission  systems  experienced  nine 
problems  that  interrupted  or  delayed  LCS  MCM  activities. 
These  problems  included  MH-60S  rotor  blade  delamination, 
an  MH-60S  power  distribution  unit  failure,  a  broken  relief 
valve  on  an  MH-60S  hydraulic  reservoir,  multiple  AMCM 
mission  kit  failures  that  required  the  MH-60S  to  return  to  port 
for  repairs,  and  an  AMNS  neutralizer  that  failed  to  launch 
when  commanded.  The  launch  failure  would  have  required 
the  aircrew  to  jettison  the  launch  and  handling  system  if  live 
rounds  (operational  assets)  been  employed.  As  a  result,  LCS  2 
demonstrated  sustained  MH-60S  operations  lasting  more  than 
one  week  just  once  during  TECHEVAL. 

•  The  Airborne  Mine  Detection  System  (ALMDS)  does 
not  meet  Navy  detection/classification  requirements.  In 
particular,  the  system  does  not  meet  the  Navy’s  requirements 
for  minimum  probability  of  detection  and  classification  in 
all  depth  bins  or  for  the  average  probability  of  detection  and 
classification  in  all  conditions  over  a  region  of  the  water 
column  that  extends  from  the  surface  to  a  reduced  maximum 
depth  requirement.  When  the  system  and  operator  detect 
and  classify  a  smaller  percentage  of  mines  than  predicted 

by  fleet  planning  tools,  the  MCM  commander  will  likely 
underestimate  the  residual  risk  to  transiting  ships  following 
clearance  operations.  In  some  conditions,  the  system  also 
generates  a  large  number  of  false  classifications  (erroneous 
indications  of  mine-like  objects)  that  can  delay  near-surface 
minehunting  operations  until  conditions  improve  or  slow 


mine  clearance  efforts  because  of  the  need  for  additional 
search  passes  to  reduce  the  number  of  false  classifications. 

In  favorable  conditions,  such  as  those  observed  during 
TECHEVAL,  detection  performance  meets  the  Navy’s 
requirements  and  tactics,  techniques,  and  procedures  have 
been  successful  in  reducing  false  classifications  to  the  Navy’s 
acceptable  limits. 

•  The  current  increment  of  the  AMNS  cannot  neutralize  moored 
mines  above  a  prescribed  operating  ceiling,  which  will 
preclude  neutralizing  most  of  the  mines  expected  in  some 
likely  threat  scenarios;  an  Explosive  Ordinance  Disposal  Team 
provided  by  another  unit  must  be  used.  Within  its  operating 
range,  AMNS  performance  is  frequently  degraded  by  the  loss 
of  fiber-optic  communications  between  the  aircraft  and  the 
neutralizer.  The  system  has  experienced  loss  of  fiber-optic 
communications  in  a  wide  range  of  operationally  relevant 
operating  conditions,  including  those  that  are  relatively 
benign.  Although  the  Program  Office  has  stated  that  it  intends 
to  develop  an  improved  AMNS  to  extend  its  depth  range  and 
potentially  improve  performance  in  coarse  bottom  conditions 
and  higher  currents,  none  of  these  efforts  are  funded  and  the 
Navy  is  considering  other  alternatives. 

System 

•  The  MH-60S  is  a  medium  lift  ship-based  helicopter 
manufactured  in  three  variants  (blocks)  that  are  derived  from 
the  Army  UH-60L  Blackhawk. 

•  All  three  blocks  share  a  common  cockpit,  avionics,  flight 
instrumentation,  and  power  train  with  the  MH-60R. 

•  Installed  systems  differ  by  block  based  on  mission: 

-  Block  1,  Fleet  Logistics  -  precision  navigation  and 
communications,  maximum  cargo  or  passenger  capacity. 


MH-60S  249 


F Y1  5  NAVY  PROGRAMS 


-  Block  2 A/B,  AMCM  System  -  AMCM  system  operator 
workstation;  a  tether/towing  system  and  the  two  MCM 
systems  currently  under  development;  ALMDS  for 
detection  and  classification  of  near-surface  mines;  and  the 
AMNS  for  neutralization  of  in  volume  and  bottom  mines. 
Any  Block  2  or  subsequent  aircraft  (e.g.,  Block  3  A/B 
aircraft)  can  be  an  AMCM  aircraft. 

-  Block  3  A,  Armed  Helicopter  -  20  mm  Gun  System, 
forward-looking  infrared  with  laser  designator,  crew  served 
side  machine  guns,  dual-sided  HELLFIRE  air-to-ground 
missiles,  the  2.75-inch  family  of  rockets,  and  defensive 
electronic  countermeasures. 

-  Block  3B,  Armed  Helicopter  -  adds  a  tactical  datalink 
(Link  16)  to  Block  3  A  capabilities. 

Mission 

The  Maritime  Component  Commander  can  employ  variants  of 

MH-60S  to  accomplish  the  following  missions: 


•  Block  1  -  Vertical  replenishment,  internal  cargo  and  personnel 
transport,  medical  evacuation,  Search  and  Rescue,  and  Aircraft 
Carrier  Plane  Guard 

•  Block  2  -  Detection,  classification,  identification,  and/or 
neutralization  of  sea  mines,  depending  on  the  specific  AMCM 
systems  employed  on  the  aircraft 

•  Block  3  -  Combat  Search  and  Rescue,  Surface  Warfare, 
Aircraft  Carrier  Plane  Guard,  Maritime  Interdiction 
Operations,  and  Special  Warfare  Support 

Major  Contractors 

•  Sikorsky  Aircraft  Corporation  -  Stratford,  Connecticut 

•  Lockheed  Martin  Mission  System  and  Sensors  -  Owego,  New 
York 


Activity 

•  Having  completed  the  land-based  phase  of  an  operational 
assessment  of  the  AMNS  in  3QFY14  with  the  MH-60S 
helicopter  operating  from  Naval  Air  Station,  Oceana, 

Virginia,  the  Navy  conducted  the  ship-based  phase  aboard 
USS  Independence  (LCS  2)  in  1QFY15  during  Increment  1 
MCM  mission  package  developmental  testing.  The  ship-based 
phase  of  the  assessment  focused  on  shipboard  integration  and 
the  system’s  operational  suitability,  but  was  also  able  to  collect 
limited  effectiveness  data. 

•  The  Navy  also  completed  the  ship-based  phase  of  an  ALMDS 
operational  assessment  in  1QFY15  aboard  LCS  2  during 
Increment  1  MCM  mission  package  developmental  testing 

in  accordance  with  the  DOT&E-approved  test  plan.  The  test 
collected  limited  data  to  examine  system  effectiveness  and  the 
shipboard  suitability  of  the  MH-60S  helicopter  equipped  with 
the  ALMDS. 

•  From  April  through  August  2015,  the  Navy  conducted 
TECHEVAL  of  the  Independence  variant  LCS  and  Increment 
1  MCM  mission  package,  including  the  MH-60S  and  AMCM 
systems,  aboard  LCS  2.  Although  the  Navy  originally 
planned  to  complete  the  test  in  June  2015  and  then  complete 
operational  testing  in  FY15,  problems  with  failures  of 
seaframe  and  MCM  systems  caused  the  testing  to  be  extended. 
The  Navy  has  delayed  the  operational  testing  until  the  spring 
of  2016,  at  the  earliest. 

•  in  May  2015,  the  Navy  conducted  AMNS  medium  current 
developmental  testing  from  a  surrogate  platform  in  the 
Atlantic  Ocean  near  the  South  Florida  Test  Facility.  The 
Navy  explored  alternative  tactics  and  collected  data  to  inform 
possible  system  improvements. 

•  in  June  2015,  the  Navy  commenced  ALMDS  and 
AMNS  cybersecurity  operational  testing  concurrently 
with  LCS  2  cybersecurity  testing  in  accordance  with  the 


DOT&E-approved  test  plan.  The  initial  phase  of  the 
cybersecurity  operational  test,  a  Cooperative  Vulnerability 
and  Penetration  Assessment  was  completed  in  July  2015,  but 
did  not  include  the  MH-60S.  The  final  phase  of  the  test,  an 
Adversarial  Assessment,  is  on  hold  pending  a  Navy  decision 
on  the  readiness  of  the  Increment  1  MCM  mission  package 
and  Independence  variant  LCS  for  operational  testing. 

•  In  November  2015,  DOT&E  provided  the  USD(AT&L),  the 
Assistant  Secretary  of  the  Navy  for  Research  Development 
and  Acquisition,  and  the  Program  Executive  Officer  for 
Littoral  Combat  Ships  a  classified  assessment  of  the 
performance  of  the  Independence  variant  LCS  and  Increment  1 
MCM  mission  package.  DOT&E  based  the  assessment  on  the 
data  collected  during  the  TECHEVAL  and  earlier  periods  of 
developmental  and  operational  testing. 

Assessment 

•  DOT&E  concluded  in  a  November  2015  memorandum  to 
USD(AT&L)  and  the  Navy,  based  on  the  testing  conducted 
to  date,  that  an  LCS  employing  the  current  MCM  mission 
package  would  not  be  operationally  effective  or  operationally 
suitable  if  the  Navy  called  upon  it  to  conduct  MCM  missions 
in  combat.  Three  of  the  seven  primary  shortcomings 
supporting  this  conclusion  are  attributed,  at  least  in  part,  to  the 
MH-60S  with  AMCM  systems: 

-  Critical  MCM  systems  are  not  reliable. 

-  Minehunting  capabilities  are  limited  in  other-than-benign 
environmental  conditions. 

-  The  AMNS,  by  design,  cannot  neutralize  most  of  the  mines 
in  the  Navy’s  threat  scenarios;  an  Explosive  Ordinance 
Disposal  Team  provided  by  another  unit  must  be  used. 

•  During  TECHEVAL,  the  MH-60S  and  its  associated  AMCM 
mission  kit  and  mission  systems  experienced  nine  problems 
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that  interrupted  or  delayed  LCS  MCM  activities.  These 
problems  included  MH-60S  rotor  blade  delamination,  an 
MH-60S  power  distribution  unit  failure,  a  broken  relief  valve 
on  an  MH-60S  hydraulic  reservoir,  multiple  AMCM  mission 
kit  failures  that  required  the  MH-60S  to  return  to  port  for 
repairs,  and  an  AMNS  neutralizer  that  failed  to  launch  when 
commanded.  The  launch  failure  would  have  required  the 
aircrew  to  jettison  the  launch  and  handling  system  if  live 
rounds  (operational  assets)  been  employed.  As  a  result,  LCS  2 
demonstrated  sustained  MH-60S  operations  lasting  more  than 
one  week  just  once  during  TECHEVAL. 

-  On  eight  occasions,  LCS  2  conducted  MH-60S  operations 
for  two  days  or  less  before  needing  repairs  that  in  many 
cases  required  the  ship  or  helicopter  to  return  to  port  for 
spare  parts  or  repairs.  In  one  case,  after  returning  to  port, 
the  Navy  elected  to  replace  a  helicopter  embarked  aboard 
LCS  and  in  need  of  repairs  rather  than  repair  it. 

-  In  total,  during  132  days  of  TECHEVAL,  the  LCS  2 
Aviation  Detachment  employed  2  MH-60S  helicopters  for 
141  flight  hours. 

-  Considering  only  the  58  days  underway,  LCS  2  was 
ALMDS-mission  capable  for  16  days,  AMNS-mission 
capable  for  26  days,  and  not  capable  of  conducting  the 
planned  AMCM  mission  for  1 6  days  due  to  helicopter, 
tow  cable,  and  computer  problems.  Nearly  all  the  lost 
AMCM  mission  days  occurred  in  the  AMNS  configuration. 
This  is  not  surprising  given  that  the  AMNS  mission  is 
more  stressing  on  the  MH-60S  and  its  AMCM  mission  kit 
because  of  the  need  to  lower  the  loaded  AMNS  launch  and 
handing  system  into  the  water  and  retrieve  it  at  least  once 
per  sortie. 

-  The  MH-60S  aircrew  employed  2  ALMDS  pods  to  search 
for  mines  for  33  hours  and  3  AMNS  launch  and  handling 
systems  to  launch  107  inert  neutralizers  against  66  targets. 

•  The  ALMDS  does  not  meet  Navy  detection/classification 
requirements  in  all  depth  bins  or  for  the  average  probability 
of  detection  and  c  lassification  in  all  conditions  over  a  region 
of  the  water  column  that  extends  from  the  surface  to  a 
reduced  maximum  depth  requirement.  When  the  system  and 
operator  detect  and  classify  a  smaller  percentage  of  mines 
than  predicted  by  fleet  planning  tools,  the  MCM  commander 
will  likely  underestimate  the  residual  risk  to  transiting 
ships  following  clearance  operations.  To  account  for  this 
uncertainty,  the  Navy  might  find  it  necessary  to  conduct 
minesweeping  operations.  However,  the  Navy  does  not 
plan  to  include  the  mechanical  minesweeping  capability  that 
would  be  required  in  the  MCM  mission  package.  In  some 
conditions,  the  ALMDS  also  generates  a  large  number  of  false 
classifications  (erroneous  indications  of  mine-like  objects)  that 
can  delay  near-surface  minehunting  operations  until  conditions 
improve  or  slow  mine  clearance  efforts  because  of  the  need 
for  additional  search  passes  to  reduce  the  number  of  false 
classifications.  In  favorable  environmental  conditions,  such 
as  those  observed  during  TECHEVAL,  detection  performance 
meets  the  Navy’s  requirements  and  the  new  multi-pass  tactic 
has  been  successful  in  reducing  false  classifications  to  the 


Navy’s  acceptable  limits  at  the  cost  of  requiring  more  search 
and  identification  time. 

•  The  current  increment  of  the  AMNS  cannot  neutralize  moored 
mines  above  a  prescribed  operating  ceiling,  which  will 
preclude  neutralizing  most  of  the  mines  expected  in  some 
likely  threat  scenarios;  an  Explosive  Ordinance  Disposal  Team 
provided  by  another  unit  must  be  used.  Within  its  operating 
range,  AMNS  performance  is  frequently  degraded  by  the  loss 
of  fiber-optic  communications  between  the  aircraft  and  the 
neutralizer.  The  system  has  experienced  loss  of  fiber-optic 
communications  in  a  wide  range  of  operationally  relevant 
operating  conditions,  including  those  that  are  relatively  benign. 
Although  the  Program  Office  has  stated  that  it  intends  to 
develop  an  improved  AMNS  to  extend  its  depth  range  and 
potentially  improve  performance  in  coarse  bottom  conditions 
and  higher  currents,  none  of  these  efforts  are  funded  and  the 
Navy  is  considering  other  alternatives. 

-  AMNS  Increment  1  cannot  neutralize  near  surface  mines 
because  of  safety  interlocks  designed  to  protect  the 
helicopter  and  crew  from  exposure  to  fragments,  surge,  and 
blast  that  might  result  from  mine  detonation. 

-  During  the  shore -based  phase  of  an  operational  assessment 
completed  in  2014,  the  system  and  its  operators  were 
unable  to  achieve  the  Navy’s  requirement  for  mine 
neutralization  success  in  realistic  conditions.  Frequent 
loss  of  fiber-optic  communications  between  the  aircraft 
and  the  neutralizer  was  the  primary  cause  of  unsuccessful 
attack  runs.  The  Navy  attributed  the  failures  to  the 
bottom  composition  even  though  the  bottom  conditions 
experienced  in  the  test  area  were  not  significantly  different 
from  those  expected  in  some  potential  operating  areas. 

-  During  TECHEVAL,  which  was  conducted  in  favorable 
environmental  conditions  against  a  narrower  segment  of 
mine  threats,  the  Navy  observed  higher  probabilities  of 
AMNS  neutralization  success  than  observed  during  the 
2014  operational  assessment.  However,  preliminary  results 
indicate  performance  is  consistent  with  previous  results  in 
the  same  environment. 

-  Following  developmental  testing  in  high-current 
environments  in  2013,  Navy  Air  Test  and  Evaluation 
Squadron  Twenty  One  concluded  that  the  AMNS 
neutralizer,  as  currently  designed,  is  ineffective  in  swift 
water  currents.  Although  the  Navy  completed  additional 
developmental  testing  in  2015,  the  Navy’s  testing  has  not 
characterized  system  performance  under  operationally 
realistic  conditions  in  even  moderate  currents  that  might  be 
encountered  in  potential  operating  areas. 

•  Consistent  with  the  concept  of  operations,  the  LCS  is  reliant 
on  shore -based  support  for  assistance  with  diagnosis  and  repair 
of  MCM  mission  systems  including  ALMDS  and  AMNS. 

The  mission  package  detachment  lacks  the  wherewithal  to 
handle  anything  beyond  relatively  uncomplicated  preventive 
maintenance  and  minor  repairs.  Thus,  when  ALMDS  and 
AMNS  failures  occur,  the  Navy  assumes  that  in  most  cases 
these  systems  will  be  replaced  by  embarked  or  shore-based 
spares. 
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•  Although  TECHEVAL  is  intended  to  be  developmental  testing 
in  nature,  the  test  was  designed  to  integrate  the  test  objectives 
of  both  developmental  and  operational  test  communities. 
DOT&E  personnel  observed  the  testing  aboard  LCS  2.  If 

the  Navy  elects  to  continue  with  the  same  system  hardware 
and  software  configurations,  DOT&E  and  the  Navy  will 
use  the  resulting  data  to  supplement  data  collected  during 
the  operational  test.  If  the  Navy  decides  to  go  forward  to 
operational  testing  with  a  new  system,  integrated  test  data 
collected  in  FY15  may  not  be  representative  of  the  system  the 
Navy  intends  to  field,  and  the  Navy  might  need  to  repeat  some 
portions  of  TECHEVAL  to  provide  the  requisite  data. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  partially 
addressed  the  FY11  recommendation  to  investigate  solutions 
and  correct  the  ALMDS  False  Classification  Density  and 
reliability  deficiencies  prior  to  IOT&E.  The  Navy  has  partially 
addressed  the  FY12  recommendation  to  assess  corrections 
made  to  resolve  previously  identified  Multi-spectral  Targeting 
System  (MTS)  deficiencies  by  conducting  FOT&E.  The  Navy 
has  not  acted  or  has  yet  to  complete  action  on  FY13  and  FY14 
recommendations  to: 

1.  Complete  comprehensive  survivability  studies  for  MH-60S 
employing  the  20  mm  Gun  System  and  the  2.75”  Unguided 
Rocket  Launcher. 

2.  Conduct  comprehensive  live  fire  lethality  testing 
of  the  HELLFIRE  missile  against  a  complete  set  of 
threat-representative  small  boat  targets. 

3.  Correct  the  tracking  deficiencies  in  the  MTS  and  conduct 
appropriate  FOT&E  in  order  to  satisfactorily  resolve  the 
Surface  Warfare  Critical  Operational  Issue. 


4.  Complete  comprehensive  IOT&E  on  the  2.75”  Unguided 
Rocket  and  Advanced  Precision  Kill  Weapon  System 

II  (APKWS  II)  to  resolve  the  Surface  Warfare  Critical 
Operational  Issue  not  resolved  in  limited  assessments 
of  system  performance  provided  in  Quick  Reaction 
Assessments  against  small  boat  threats. 

5.  Test  the  Surface  Warfare  mission  capability  of  MH-60S 
helicopter  equipped  with  MTS  and  the  HELLFIRE  missile 
throughout  the  operational  mission  environment  in  FOT&E 
and  LFT&E. 

6.  Complete  vulnerability  studies  for  MH-60S  employing  the 
LAU-61G/A  Digital  Rocket  Launcher  armed  with  APKWS 
II  rockets. 

7.  Conduct  comprehensive  lethality  testing  of  the  LAU-61G/A 
Digital  Rocket  Launcher  armed  with  APKWS  II  rockets 
against  a  complete  set  of  threat-representative  small  boat 
targets. 

8.  Correct  AMCM  mission  kit  reliability  issues  that  limit 
AMNS  mission  availability  identified  during  the  operational 
assessment. 

9.  Develop  corrective  actions  to  eliminate  early  termination 
fiber-optic  communications  losses  observed  in  the  AMNS 
operational  assessment. 

10.  Conduct  AMNS  current  testing  from  MH-60S. 

•  FY15  Recommendation. 

1.  The  Navy  should  provide  LCS  with  a  mine  neutralization 
capability  in  water  depths  above  the  current  AMNS 
operating  ceiling. 
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Mine  Resistant  Ambush  Protected  (MRAP) 
Family  of  Vehicles  (FoV)  -  Marine  Corps 


Executive  Summary 

•  The  U.S.  Marine  Corps  will  retain  2,510  Mine  Resistant 
Ambush  Protected  (MRAP)  vehicles  in  its  enduring  fleet 
including  300  Cougar  Category  (CAT)  II A1  variants. 

•  Live  fire  testing  conducted  in  FY15  indicates  that  the  Cougar 
CAT  II A 1  variant  with  seat  survivability  upgrades  will  meet 
its  required  level  of  performance. 

•  The  Marine  Corps  is  planning  to  retrofit  all  retained  Cougar 
variants  with  egress  upgrades.  Live  fire  testing  of  these 
upgrades  will  commence  in  2QFY16,  and  the  first  test  will 
be  conducted  on  a  CAT  II A 1  variant  with  both  the  seat  and 
egress  upgrades. 

System 

•  MRAP  Family  of  Vehicles  (FoV)  consists  of  medium-armored, 
all-wheel  drive,  tactical  wheeled  vehicles  designed  to  provide 
protected  mobility  for  personnel  in  a  threat  environment. 
Relative  to  the  High  Mobility  Multipurpose  Wheeled  Vehicle, 
MRAPs  provide  improved  crew  protection  and  vehicle 
survivability  against  current  battlefield  threats,  such  as 

IEDs,  mines,  small  arms  fire,  rocket-propelled  grenades,  and 
explosively  formed  penetrators. 

•  Based  on  a  recommendation  in  DOT&E’s  2010  assessment 
of  the  MRAP  FoV  to  improve  the  seats  in  the  Cougar  A 1 
vehicles,  the  Marine  Corps  intends  to  retrofit  the  Cougar  Cat  II 
A1  vehicles  with  a  seat  survivability  upgrade  (SSU)  kit.  The 
SSU  is  primarily  a  redesign  of  the  rear  crew  compartment  of 
the  Cougar,  focusing  on  improved  seating  for  vulnerability 
reduction,  safety,  and  human  factors  integration.  A  Cat  II A 1 
Cougar  with  the  SSU  can  carry  10  Marines  and  1  gunner. 

•  Currently,  the  Marine  Corps  will  retain  2,510  MRAP  vehicles 
in  its  enduring  fleet,  including  300  CAT  II A1  vehicles.  The 


Cougar  Category  (CAT)  II  A1 


Marine  Corps  will  remain  the  Primary  Inventory  Control 
Activity  for  all  Cougar  platforms,  including  those  vehicles 
divested  to  the  Navy  and  Air  Force. 

Mission 

Commanders  will  employ  Marine  units  equipped  with  the  MRAP 
Cougar  CAT  II A 1  to  conduct  mounted  patrols,  reconnaissance, 
communications,  and  command  and  control  missions  in  a  threat 
environment. 

Major  Contractor 

General  Dynamics  Land  Systems  -  Ladson,  South  Carolina 


Activity 

•  The  Marine  Corps  plans  to  retrofit  a  total  of  263  Marine  Corps, 
41  Air  Force,  and  114  Navy  CAT  II A1  vehicles  with  the  SSU 
upgrade.  The  SSU  upgrade  is  designed  to  achieve  Capabilities 
Production  Document  1 . 1  objective-level  force  protection 
against  underbody  and  under-wheel  blast  mines.  The  Marine 
Corps  will  also  install  egress  upgrades  on  all  of  these  vehicles, 
as  well  as  other  retained  Cougar  variants. 

•  From  August  2014  through  March  2015,  the  Marines  Corps 
conducted  three  of  four  planned  live  fire  tests  of  the  CAT  II 
A1  Cougar  with  the  SSU.  The  fourth  test,  projected  to  occur 
in  2QFY16,  will  be  conducted  with  both  SSU  and  egress 


upgrades  when  egress  modifications  are  available  to  be 
outfitted  on  the  test  asset. 

•  The  Marine  Corps  conducted  all  testing  in  accordance  with  a 
DOT&E-approved  test  plan. 

Assessment 

•  Live  fire  testing  to  date  indicates  that  the  Cougar  CAT  II A 1 
with  the  SSU  meets  its  contract  specifications  and  provides 
force  protection  at  the  required  Capability  Production 
Document  1.1  objective  level.  However,  additional  planned 
live  fire  testing  remains  to  determine  the  response  of  the 
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Cougar  Cat  II A 1  when  outfitted  with  the  both  the  SSU  and 
egress  modifications. 

•  The  results  from  the  legacy  Cougar  live  fire  test  program  (as 
found  in  the  2010  DOT&E  report  on  the  original  MRAP  FoV 
Cougar  vehicles)  relative  to  other  tested  threats  such  as  IEDs, 
indirect  fire,  small  arms  fires,  rocket-propelled  grenades,  and 
explosively-formed  penetrators  are  applicable  to  the  Cougar 
CAT  II A 1  with  the  SSU. 


Recommendations 

•  Status  of  Previous  Recommendations.  The  program  is 
making  progress  implementing  the  previous  recommendations 
regarding  upgrading  the  seats  in  the  Cougar  CAT  II A 1 
vehicles. 

•  FY15  Recommendations.  None. 
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Mobile  Landing  Platform  (MLP)  Core  Capability  Set 
(CCS)  (Expeditionary  Transfer  Dock)  and  Afloat  Forward 
Staging  Base  (AFSB)  (Expeditionary  Mobile  Base) 


Executive  Summary 

•  From  August  25  through  November  3,  2014,  the  Navy 
conducted  the  MLP  (CCS)  IOT&E. 

•  On  July  6,  2015,  DOT&E  published  a  classified,  combined 
IOT&E  and  LFT&E  report  and  assessed  that  MLP  (CCS): 

-  Is  capable  of  transiting  the  required  9,500  nautical  miles  at 
15  knots  unrefueled. 

-  Can  land  and  launch  Landing  Craft  Air  Cushion  (LCAC) 
vehicles  through  Sea  State  3. 

-  Is  operationally  effective,  achieving  the  primary  timed 
requirement  of  enabling  the  transfer  ashore  of  a  reinforced 
rifle  company’s  equipment  within  12  hours  from 

25  nautical  miles  out  to  sea. 

-  Can  operate  skin-to-skin,  to  include  vehicle  transfer 
through  Sea  State  3,  with  both  the  USNS  Bob  Hope 

( Bob  Hope  class)  and  USNS  Dahl  ( Watson  class)  Large 
Medium  Speed  Roll-on  roll-off  (LMSR)  ships. 

-  Is  not  operationally  effective  in  its  interoperability  with  the 
Joint  High  Speed  Vessel  (JHSV)  since  it  is  only  feasible 
when  done  in  Sea  State  less  than  1  (JHSV’s  ramp  is  limited 
to  significant  wave  height  of  no  more  than  0.1  meters) 
conditions,  which  are  normally  only  found  in  protected 
harbors.  However,  when  tested  in  a  more  operationally 
relevant  open-ocean  environment,  the  JHSV  ramp  suffered 
a  casualty. 

-  Has  satisfactory  cybersecurity  with  no  significant 
vulnerabilities.  Even  if  a  cyber-adversary  gained  access, 
overall  ship’s  mission  disruption  would  be  minimal. 

•  The  MLP  program  did  not  conduct  any  major  live  fire  test 
events  during  FY15.  The  Navy  plans  to  conduct  the  Total 
Ship  Survivability  Trial  to  obtain  data  for  recoverability 
analysis  on  the  MLP  Afloat  Forward  Staging  Base  (AFSB) 
variant  in  FY 16.  The  Navy  plans  to  issue  the  final 
Survivability  Assessment  Report  in  FY17. 

System 

•  The  MLP  (CCS)  is  now  called  the  Expeditionary  Transfer 
Dock,  and  the  MLP  (AFSB)  is  now  called  Expeditionary 
Mobile  Base. 

•  The  MLP  is  a  modified  heavy-lift  ship  based  on  the  British 
Petroleum  Alaska  class  oil  tanker,  procured  by  the  Navy  to  use 
float-on/float-off  technology. 

•  The  Navy  developed  the  MLP  to  host  multiple  mission  sets, 
operate  from  international  waters,  and  persist  for  extended 
periods  providing  a  capability  unencumbered  by  geo-political 
constraints  to  meet  strategic  goals. 


Afloat  Forward  Staging  Base  (AFSB) 


•  The  MLP  (CCS)  requires  34  Military  Sealift  Command 
(MSC)-contracted  mariners  to  operate  and  maintain  the 
vessels.  MLP  AFSB  requires  34  MSC  civilian  mariners  and 
101  permanent  military  crew  to  operate  and  maintain  the 
vessels.  MSC  will  serve  as  MLP  Life  Cycle  Managers. 

•  The  Navy  delivered  two  MLPs  with  the  CCS  mission 
capabilities  (hulls  1  and  2),  and  plans  to  deliver  three  MLPs 
with  AFSB  mission  capability  (hulls  3,  4,  and  5). 

•  The  MLP  (CCS): 

-  Supports  Mobile  Prepositioning  Force  operations  by 
facilitating  at-sea  transfer  and  delivery  of  pre -positioned 
assets  to  units  ashore. 

-  Consists  of  a  vehicle-staging  area  (raised  vehicle  deck), 
vehicle  transfer  ramp  (VTR),  large  mooring  fenders,  an 
emergency-only  commercial  helicopter  operating  spot,  and 
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three  LCACs  lanes/operating  spots  with  barriers,  catwalk, 
lighting,  wash  down,  and  fueling  services. 

-  Is  equipped  with  a  crane  and  work  boat  for  placing  of 
fenders  used  for  skin-to-skin  operations  with  an  LMSR  or 
JHSV. 

-  Interfaces  with  the  LMSR  ships  and  LCACs  and  intends 
to  interface  with  the  follow-on  Ship-to-Shore  Connectors 
to  permit  off-load  and  transfer  ashore  of  military  vehicles 
ranging  from  High  Mobility  Multi-purpose  Wheeled 
Vehicles  to  battle  tanks  (M1A2). 

-  Is  classified  as  a  non-combatant  with  a  limited 
self-protection  capability. 

•  The  MLP  (AFSB): 

-  Includes  a  two-spot  flight  deck,  hangar  facility,  helicopter 
fueling  capability,  ordnance  storage,  operation  planning 
and  work  spaces,  and  berthing  for  101  permanent 
military  crew  and  149  personnel  of  an  embarked  military 
detachment. 

-  Has  a  mission  deck  below  the  flight  deck  with  a  crane 
for  storing  and  deploying  the  various  mine-hunting  and 
clearing  equipment  used  with  the  MH-53E  helicopters; 
explosive  ordnance  demolition  boats  and  equipment  may 
also  be  stored  and  handled  on  the  mission  deck. 

-  Is  classified  as  a  non-combatant  with  a  limited 
self-protection  capability. 

-  Is  built  to  commercial  standards  with  the  ship  structure 
remaining  relatively  the  same  as  MLP  (CCS).  However, 
the  aviation  facilitates;  forward  house  Command,  Control, 
Communications,  Computers  and  Intelligence  suite; 


fueling  at-sea  station;  and  ordnance  stowage  and  handling 
components  are  built  to  U.S.  Navy  standards. 

-  Will  be  exposed  to  a  larger  number  of  threats  than  the 
MLP  (CCS)  variant;  therefore,  it  will  be  dependent  on 
other  vessels  to  provide  defense  for  more  traditional 
anti-ship  weapons  encountered  by  naval  vessels. 

Mission 

•  Combatant  Commanders  will  use  the  MLP  (CCS)  as  a  surface 
interface  between  other  Mobile  Prepositioning  Force  (future) 
squadron  ships  (such  as  LMSRs,  and  JHSVs),  connectors 
(LCACs  and  Ship-to-Shore  Connectors)  and  sea  base  to 
transfer  personnel  and  equipment  to  facilitate  delivery  ashore 
of  forces  from  the  sea. 

•  Combatant  Commanders  will  use  the  MLP  (AFSB): 

-  To  support  Airborne  Mine  Countermeasure  operations 
including  hosting  a  squadron  of  four  legacy  MH-53E 
helicopters 

-  Provide  storage  and  deploying  capabilities  of  the  various 
mine-detecting  and  clearing  equipment,  which  are  used 
with  the  helicopters 

-  To  support  explosive  ordnance  demolition  boats 

Major  Contractors 

•  MLP  base  ship  and  MLP  AFSB:  General  Dynamic’s  National 
Steel  and  Shipbuilding  Company  (NASSCO)  -  San  Diego, 
California 

•  CCS  arrangement:  Vigor  Marine  (Limited  Liability  Company) 
Shipbuilding  -  Portland,  Oregon 


Activity 

•  The  Navy  conducted  the  MLP  (CCS)  IOT&E  from 
August  25  through  November  3,  2014. 

•  DOT&E’s  FY14  Annual  Report  listed  test  activity  through 
September  2014.  This  report  outlines  the  last  IOT&E  events 
the  Navy  conducted  from  October  through  November  2014. 
The  following  test  events  were  conducted  at-sea,  primarily  in 
the  vicinity  of  Camp  Pendleton,  California: 

-  In  October  2014,  the  Navy’s  Commander,  Operational 
Test  and  Evaluation  Force  (COTF)  conducted  a  24-hour, 
15-knot  fuel  economy  trial  as  MLP  1  transited  from  the 
Seattle,  Washington,  area  to  Southern  California  waters. 

-  In  October  2014,  COTF  and  the  Marine  Corps  Operational 
Test  and  Evaluation  Activity  (MCOTEA)  successfully 
conducted  skin-to-skin  operations  through  Sea  State  3 
with  both  the  USNS  Bob  Hope  (Bob  Hope  class)  and  the 
USNS  Dahl  (Watson  class)  LMSRs,  to  include  vehicle 
transfer  and  LCAC  operations  in  various  tests. 

-  On  October  14,  2014,  COTF  and  MCOTEA  successfully 
completed  the  reinforced  rifle  company  vehicle  transfers 
via  LCAC  in  a  timed  event.  Although  MLP  1  was  not 
positioned  25  nautical  miles  from  shore,  the  LCACs 
transited  a  route  of  25  nautical  miles  both  to  shore  and 


then  back  to  MLP  1  for  the  next  load  to  support  the  timed 
transfer  requirement. 

-  On  October  29,  2014,  COTF  and  MCOTEA  attempted  the 
open-ocean,  day  and  night,  interface  test  mooring  of  the 
JHSV  to  the  MLP  (CCS)  vessel.  USNS  Millinocket  (JHSV 
3)  moored  skin-to-skin  with  USNS  Montford  Point  (MLP  1 
CCS)  and  a  Marine  Corps  vehicle  transited  back  and  forth 
during  daylight.  An  earlier  mooring  line  problem,  which 
occurred  during  a  previous  test  (June  2014),  was  resolved 
but  a  JHSV  ramp  casualty  precluded  completion  of  the 
planned  test. 

-  In  October  2014,  COTF  conducted  a  Critical  System 
Maintenance  Review  with  the  ship’s  company  to  assist  in 
evaluating  suitability  of  both  maintenance  and  logistics  for 
the  ship  class. 

-  On  November  3,  2014,  COTF  and  MCOTEA  conducted 
a  limited  self-defense  drill  (no  targets  engaged)  and  a 
Structural  Test  Fire  event  that  verified  fields-of-fire  and  the 
0.50  caliber  machine  gun  mount  structure  suitability. 

USNS  Lewis  B.  Puller  (MLP  3  AFSB)  launched  in 

November  2014;  completed  builder  trials  in  April  2015 

and  acceptance  trials  in  May  2015;  delivered  in  June  2015; 
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and  conducted  transit  from  San  Diego,  California,  to 
Norfolk,  Virginia,  from  August  to  October  2015. 

•  The  MLP  program  did  not  conduct  any  major  LFT&E  events 
during  FY15.  The  Navy  plans  to  conduct  the  Total  Ship 
Survivability  Trial  to  obtain  data  for  recoverability  analysis 
of  the  MLP  AFSB  in  FY16.  The  Navy  plans  to  issue  the  final 
Survivability  Assessment  Report  in  FY17. 

•  In  July  2015,  DOT&E  published  a  classified,  combined 
IOT&E  and  LFT&E  report  on  the  MLP  (CCS). 

•  The  Navy  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Maser  Plan  and  test 
plan. 

Assessment 

•  Vehicle  transfer  at-sea  with  JF1SV  moored  sldn-to-skin 
with  MLP  (CCS)  is  not  advised  (mooring  with  JF1SV  is  a 
secondary  mission  for  MLP).  The  0.1  meter  significant  wave 
height  (Sea  State  1)  limitation  of  the  JHSV  ramp  precludes 
operationally  relevant  at-sea  vehicle  transfers  with  MLP 
(CCS).  Initial  in-harbor  testing  was  successful  for  vehicle 
transfers;  however,  while  JF1SV  was  moored  skin-to-skin  in 
calm  seas  with  MLP  (CCS),  several  mooring  lines  parted, 
precluding  completion  of  the  test  event.  During  the  second 
test  event  at-sea,  the  mooring  line  issue  was  resolved,  but 
the  JF1SV  ramp  suffered  a  casualty  and  vehicle  transfer  had 
to  be  stopped.  Sea  state  for  the  at-sea  test  was  Sea  State  3 
(significant  wave  height  up  to  1.25  meters)  although  MLP 
(CCS)  created  a  lee,  effectively  reducing  the  seas  to  just  above 
Sea  State  1  for  the  skin-to-skin  mooring  with  JHSV. 

•  The  MLP  (CCS)  is  operationally  effective  provided  that 
operations  are  conducted  in  a  safe,  well-guarded  area  with 
Sea  State  3  conditions  (equates  to  significant  wave  height  up 
to  1.25  meters).  When  the  MLP  was  positioned  25  nautical 
miles  from  the  LCAC  shore  landing  site,  it  met  its  timed 
transfer  requirement,  enabling  Marine  Corps  equipment  for 
a  reinforced  rifle  company  to  be  moved  to  shore  in  less  than 
12  hours. 

•  For  operational  scenarios  that  include  Amphibious  Assault 
Vehicles  (AAVs)  independently  moving  to  shore,  the  MLP 
(CCS)  demonstrated  it  could  launch  AAVs  from  within 

5  nautical  miles  of  the  shore;  launching  AAVs  that  close  to  the 
shore  is  unlikely  to  be  feasible  in  major  combat.  However,  in 
that  particular  case,  DOT&E  estimates  the  transfer  of  a  full 
reinforced  rifle  company’s  equipment  set  would  span  52  hours 
and  49  minutes,  owing  to  the  time  needed  to  move  MLP  (CCS) 
from  25  nautical  miles  to  within  5  nautical  miles  from  shore. 

•  Based  on  a  24-hour  fuel  economy  trial,  DOT&E  estimates 
MLP  (CCS)  to  have  an  un-refueled  range  of  greater  than 
12,000  nautical  miles,  exceeding  the  9,500-nautical  mile 
requirement. 

•  The  Navy  demonstrated  skin-to-skin  operations  and  vehicle 
transfer  through  Sea  State  3  with  both  the  USNS  Bob  Hope 
(Bob  Hope  class)  and  USNS  Dahl  ( Watson  class)  LMSRs. 

•  While  conducting  vehicle  transfers  between  MLP  (CCS) 
and  LMSRs,  the  mild  side-to-side  rolling  of  the  ships  while 
moored  skin-to-skin  caused  twisting  of  the  VTR  that  must 


be  monitored.  Devices  for  monitoring  the  VTR  twist  were 
temporarily  installed  for  testing.  The  sensitivity  of  the  VTR  to 
twisting  warrants  a  permanent  system. 

•  Accelerated  wear  of  the  Main  Diesel  Generators  is  expected 
due  to  prolonged  electrical  under-loading. 

•  Ship  service  electric  power  suffered  from  power  spikes  due  to 
inadequate  electronic  grooming. 

•  The  local  cybersecurity  test  demonstrated  that  the  network’s 
host-based  security  system  stopped  most  of  COTF’s 
cyber-attacks  against  unclassified  and  classified  networks.  As 
the  ship’s  networks  are  not  connected  to  the  ship’s  critical 
systems,  the  loss  of  either  unclassified  or  classified  networks 
during  operations  would  be  an  inconvenience,  but  would  not 
hinder  the  ship’s  ability  to  conduct  its  mission  since  it  has 
communication  backups,  including  radios  and  standalone 
satellite  phones. 

•  During  the  remote  reconnaissance  and  cyber-attack  evaluation, 
COTF  was  unable  to  gain  a  foothold  on  the  MLP  1  networks 
with  the  toolset  used  for  these  assessments.  However,  the  test 
did  not  explore  the  vulnerability  of  the  ship  to  very  advanced 
cyber  threats  due  to  security  restrictions  in  place  during  the 
time  of  the  test. 

•  MLP  (CCS)  is  survivable  only  if  used  in  benign  and/or 
permissible  environments. 

-  MLP  (CCS)  is  designed  and  built  to  commercial  standards 
that  do  not  include  hull  and  equipment  hardening  or 
personnel  protection  features  necessary  to  survive  major 
weapon  effects. 

-  MLP  (CCS)  has  no  active  or  passive  systems  to  reduce 
susceptibility  to  enemy  weapons. 

-  The  design  has  only  limited  system  redundancy  and 
separation  to  improve  vulnerability  and  recoverability. 

•  The  effectiveness  of  area  defense  provisions  was  not  assessed 
as  part  of  this  test  and  evaluation  program. 

-  Although  the  Embarked  Security  Teams,  which  consist 
of  12  members,  are  manned  with  well-trained  individuals 
equipped  with  0.50-caliber  weapons,  they  can  only  support 
a  24-hour  day  manning  of  four  stations.  There  are  little  test 
data  to  suggest  they  provide  effective  force  protection. 

-  For  close-in  self-defense,  the  security  teams  embark  with 
their  own  weapons  and  ammunition. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  still  needs  to 
address  the  FY14  recommendation  to  re-evaluate  the  need  for 
at-sea  skin-to-skin  operations  between  MLP  (CCS)  and  JHSV. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Install  permanent  VTR  twist  sensors  and  provide  a  display 
monitor  on  the  MLP  (CCS)  ships  to  assist  the  MLP  (CCS) 
Master  during  skin-to-skin  operations. 

2.  Install  a  separate  Ship  Service  Diesel  Generator  to  minimize 
periods  of  under-loading  of  the  Main  Diesel  Generators. 

3.  Install  additional  ship  service  electrical  grooming 
equipment  to  alleviate  ship  service  power  spikes  and 
minimize  damage  to  sensitive  electronic  equipment. 
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4.  Address  the  cybersecurity  and  live  fire  issues  identified  in 
the  classified  annex  to  the  July  2015  DOT&E  combined 
IOT &E  and  LFT &E  report. 

5.  Conduct  a  robust,  self-defense  test  utilizing  live  ammunition 
and  realistic  targets  in  support  of  the  MLP  (AFSB)  IOT&E. 
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Mobile  User  Objective  System  (MUOS) 


Executive  Summary 

•  The  Navy  program  manager  conducted  a  government 
developmental  test  from  June  1  -  30,  2015,  in  preparation  for 
operational  testing. 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  the  Mobile  User  Objective  System 
(MUOS)  Multi-Service  Operational  Test  and  Evaluation 
(MOT&E)  from  October  19  through  November  20,  2015. 
COTF  is  planning  to  conduct  a  two-phased  cybersecurity 
assessment  on  the  MUOS  system  in  conjunction  with  the 
MOT&E  in  November  2015  and  2QFY16. 

•  The  program  manager  has  made  progress  integrating  the 
end-to-end  MUOS  capability.  Technical  Evaluation  results 
suggest  that  deployed  users  can  reliably  make  point-to-point 
data  and  voice  calls  and  point-to-net  data  calls.  MUOS  group 
communications,  which  allows  the  system  to  maintain  links 
between  all  users  in  group  networks,  is  not  as  reliable  and 
may  lead  to  the  loss  of  mission  information  and  situational 
awareness  for  some  users. 

•  As  of  October  19,  2015,  there  were  over  200  high-priority 
hardware  and  software  problems  remaining  on  the  system. 

The  program  manager  has  prioritized  them  and  is  fixing  as 
funding  allows;  however,  most  of  the  problems  will  not  be 
fixed  in  time  for  the  MOT&E. 

•  The  geolocation  capability  is  still  in  development  and 
therefore,  will  not  be  ready  for  testing  in  the  2015  MOT&E. 
This  capability  will  need  to  be  operationally  tested  once 

it  is  mature.  Geolocation  is  the  ability  to  locate  a  legacy 
Ultra-High  Frequency  (UHF)  electromagnetic  interferer  on  the 
ground. 

System 

•  MUOS  is  a  satellite -based  communications  network  designed 
to  provide  worldwide,  narrowband,  beyond  line-of-sight, 
point-to-point,  and  netted  communication  services  to 
multi-Service  organizations  of  fixed  and  mobile  terminal 
users.  The  Navy  designed  MUOS  to  provide  10  times  the 
throughput  capacity  of  the  current  narrowband  satellite 
communications.  The  Navy  intends  for  MUOS  to  provide 
increased  levels  of  system  availability  over  the  current 
constellation  of  UHF  Follow-On  satellites  and  to  improve 
availability  for  small,  disadvantaged  terminals. 

•  MUOS  consists  of  six  segments: 

-  The  space  segment  consists  of  four  operational  satellites 
and  one  on-orbit  spare.  Each  satellite  hosts  two  payloads: 
a  legacy  communications  payload  that  mimics  the 
capabilities  of  a  single  UHF  Follow-On  satellite  and  a 
MUOS  communications  payload. 

-  The  Ground  Transport  Segment  is  designed  to  manage 
MUOS  communication  services  and  allocation  of  radio 
resources. 


-  The  Network  Management  Segment  is  designed  to  manage 
MUOS  ground  resources  and  allow  for  government 
controlled  precedence-based  communication  planning. 

-  The  Ground  Infrastructure  Segment  is  designed  to 
provide  transport  of  both  communications  and  command 
and  control  traffic  between  MUOS  facilities  and  other 
communication  facilities. 

-  The  Satellite  Control  Segment  consists  of  MUOS 
Telemetry,  Tracking,  and  Commanding  facilities  at  the 
Naval  Satellite  Operations  Center  Headquarters  and 
Detachment  Delta. 

-  The  User  Entry  Segment  provides  a  MUOS  waveform 
hosted  on  MUOS-compatible  terminals.  The  Army’s 
Program  Manager  Tactical  Radio  is  responsible  for 
developing  and  fielding  MUOS-compatible  terminals.  The 
Air  Force  and  Navy  are  upgrading  legacy  UHF  radios  to  be 
MUOS-compatible. 

Mission 

Combatant  Commanders  and  U.S.  military  forces  deployed 
worldwide  will  use  the  MUOS  satellite  communications  system 
to  accomplish  globally  assigned  operational  and  joint  force 
component  missions,  especially  those  involving  highly  mobile 
users.  Such  missions  include  major  conventional  war;  regional 
conflicts;  search  and  rescue;  humanitarian  or  disaster  relief 
(including  severe  weather  events);  homeland  security;  homeland 
defense;  counter-narco-terrorism;  non-combatant;  evacuation 
operations;  very  important  person  travel;  strategic  airlift; 
global  mobility;  global  strike;  Intelligence,  Surveillance,  and 
Reconnaissance;  training;  logistics  support;  and  exercise  support. 
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Major  Contractors 

•  Lockheed  Martin  Space  Systems  -  Sunnyvale,  California 

•  General  Dynamics  C4  Systems  -  Scottsdale,  Arizona 


Activity 

•  The  Navy  successfully  launched  the  MUOS-3  satellite  on 
January  20,  2015.  The  satellite  is  reached  its  geosynchronous 
orbital  slot  over  the  Atlantic  Ocean  on  August  2,  2015. 

•  The  Navy  successfully  launched  the  MUOS-4  satellite 
on  September  2,  2015.  The  satellite  is  in  transit  to  the 
geosynchronous  orbital  test  slot  in  preparation  for  contractor 
testing  in  1Q-2QFY16. 

•  In  1QFY14,  MUOS-5  production  began  and  the  Navy 
anticipates  a  launch  in  3QFY16. 

•  From  January  2014  through  May  2015,  the  program 
manager  conducted  end-to-end  integration  efforts  with  the 
Army-developed  AN/PRC-155  Manpack  radio  using  the 
MUOS  waveform,  including  multiple  satellites,  the  MUOS 
Ground  Infrastructure  and  Transport  Segments,  and  the 
Network  Management  Segment. 

•  In  January  through  May  2015,  the  program  manager 
conducted  several  communications  demonstrations  including 
one  with  U.S.  Northern  Command  in  the  Arctic,  one  with  the 
U.S.  Pacific  Command  in  the  Antarctic,  and  a  risk  reduction 
demonstration  with  a  MUOS  terminal  operating  aboard  a 
U.S.  Air  Force  C-17  Globemaster  III  over  the  Pacific  Ocean. 

•  The  Navy  program  manager  conducted  a  government 
developmental  test  from  June  1  -  30,  2015  in  preparation  for 
operational  testing. 

•  COTF  conducted  the  MUOS  MOT&E  from  October  19 
through  November  20,  2015,  in  accordance  with  the 
DOT&E-approved  the  Test  and  Evaluation  Master  Plan  and 
test  plan. 

•  The  geolocation  capability  is  still  in  development  and 
therefore,  will  not  be  ready  for  testing  in  the  2015  MOT&E. 
This  capability  will  need  to  be  operationally  tested  once  it 
is  mature.  Geolocation  is  the  ability  to  locate  a  legacy  UHF 
electromagnetic  interferer  on  the  ground. 

•  COTF  is  planning  to  conduct  a  two-phased  cybersecurity 
assessment  on  the  MUOS  system  in  conjunction  with 
the  MOT&E  in  November  2015  and  2QFY16.  COTF 
conducted  the  phase-one  Cooperative  Vulnerability  and 
Penetration  Assessment  in  November  2015  and  is  planning  a 
phase-two  Adversarial  Assessment  in  2QFY16. 

•  In  September  2015,  the  program  manager  requested  a  deferral 
of  the  geolocation  capability  from  the  MOT&E. 

Assessment 

•  The  program  manager  has  made  progress  integrating  the 
end-to-end  MUOS  capability.  Technical  Evaluation  results 
suggest  that  deployed  users  can  reliably  make  point-to-point 
data  and  voice  calls  and  point-to-net  data  calls. 


•  MUOS  group  communications,  which  allow  the  system  to 
maintain  links  between  all  users  in  group  networks,  is  not  as 
reliable  and  may  lead  to  the  loss  of  mission  information  and 
situational  awareness  for  some  users. 

•  The  ability  of  MUOS  to  create,  analyze,  and  implement 
communications  plans  is  problematic.  The  system 
occasionally  freezes  when  analyzing  what  network  resources 
are  available  and  the  network  data  MUOS  produces  is 
sometimes  inaccurate.  Without  a  valid  and  accurate 
communications  plan,  the  MUOS  cannot  create  configurations 
for  all  of  the  radios  and  users  cannot  establish  communications 
with  one  another. 

•  The  fault  management  system  of  the  Network  Management 
Segment  provides  a  large  number  of  faults,  unclear  faults, 
and  erroneously  prioritized  faults  to  the  system  operators. 

This  leads  to  poor  failure  awareness  on  the  part  of  the 
system  operators.  Uncorrected  failures  could  lead  to  loss  of 
communications  for  deployed  end-users. 

•  During  developmental  test  periods,  hardware  failures  at  the 
Radio  Access  Facilities  led  to  the  loss  of  as  much  as  half  of  the 
communications  resources  on  a  single  satellite. 

•  As  of  October  19,  2015,  there  were  over  200  high-priority 
hardware  and  software  problems  remaining  on  the  system. 

The  program  manager  has  prioritized  them  and  is  fixing  as 
funding  allows;  however,  most  of  the  problems  will  not  be 
fixed  in  time  for  the  MOT&E. 

•  The  operational  testers  expect  to  find  system  deficiencies 
during  the  planned  cybersecurity  assessment  based  on  known 
cybersecurity  vulnerabilities  discovered  during  the  program 
manager’s  developmental  testing. 

•  The  geolocation  capability  is  still  in  development  and 
therefore,  will  not  be  ready  for  testing  in  the  MOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  adequately 
addressed  all  previous  recommendations. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Update  the  Test  and  Evaluation  Master  Plan  to  include 
future  testing  of  the  geolocation  capability  once  it  is  mature 
and  re-testing  any  fixes  to  problems  found  during  MOT&E. 

2.  Fund  the  MUOS  program  at  a  level  whereby  they 
can  address  the  high-priority  hardware,  software,  and 
cybersecurity  problems  in  time  for  the  next  operational  test 
event. 
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MQ-4C  Triton  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  Navy  began  an  operational  assessment  (OA)  of  the 
MQ-4C  Triton  Unmanned  Aircraft  System  (UAS)  in 
November  2015  in  support  of  a  planned  2QFY16  Milestone 
C  decision.  The  OA  is  anticipated  to  complete  in  1QFY16. 
DOT&E  will  submit  a  classified  OAreport  in  3QFY16. 

•  Developmental  flight  testing  of  the  MQ-4C  with  integrated 
mission  systems  and  the  Integrated  Functional  Capability 
(IFC)  2.2  software  began  in  April  2015,  following  a  delay  due 
to  a  lag  in  development  of  IFC  2.2.  Poor  system  stability  and 
defect  discovery  consistent  with  early  developmental  testing 
slowed  the  progress  of  developmental  testing  of  the  mission 
systems.  The  delay  in  the  development  of  IFC  2.2  and  the 
slower  than  anticipated  pace  of  IFC  2.2  developmental  testing 
postponed  the  start  of  the  OA. 

•  The  program  continues  to  pursue  a  solution  providing  traffic 
de-confliction  and  collision  avoidance  capability  (“due 
regard”)  since  stopping  the  development  of  the  Air-to-Air 
Radar  Subsystem.  The  program  intends  to  select  a  technical 
solution  in  FY16  for  delivery  after  Initial  Operational 
Capability  (IOC).  The  Navy  is  investigating  alternative 
means  of  due  regard  compliance  including  procedures  and 
other  cooperative  avoidance  systems  already  integrated  in  the 
MQ-4C  in  order  to  support  Triton  operations  at  IOC. 

System 

•  The  MQ-4C  Triton  UAS  is  an  Intelligence,  Surveillance, 
and  Reconnaissance  system-of-systems  consisting  of  the 
high-altitude,  long  endurance  MQ-4C  air  vehicle,  sensor 
payloads,  and  supporting  ground  control  stations.  The 
MQ-4C  system  is  a  part  of  the  Navy  Maritime  Patrol 
and  Reconnaissance  family-of-systems  and  will  provide 
multiple  types  of  surveillance  data  over  vast  tracks  of  ocean 
and  littoral  areas;  overland  Intelligence,  Surveillance,  and 
Reconnaissance;  signals  intelligence  and  target  acquisition 
capabilities  designed  to  complement  the  P-8A  Poseidon 
Multi-mission  Maritime  Patrol  aircraft. 

•  The  MQ-4C  air  vehicle  design  is  based  on  the  Air  Force 
RQ-4B  Global  Hawk  air  vehicle  with  significant  modifications 
that  include  strengthened  wing  structures  and  an  anti-ice  and 
de-icing  system. 

•  Mission  systems  include  a  maritime  surveillance  radar 
to  detect,  identify,  and  track  surface  targets  and  produce 
high-resolution  imagery. 

-  An  electro  optical/infrared  sensor  provides  full  motion 
video  and  still  imagery  of  surface  targets  and  the  electronic 


support  measures  system  detects,  identifies,  and  geolocates 
radar  threat  signals. 

-  An  Automatic  Identification  System  (AIS)  receiver  permits 
the  detection,  identification,  geolocation,  and  tracking 

of  cooperative  maritime  vessels  equipped  with  AIS 
transponders. 

•  Planned  future  system  upgrades  include  an  air  traffic  collision 
avoidance  radar  system  and  a  signals  intelligence  collection 
system.  Onboard  line-of-sight  and  beyond  line-of-sight 
datalink  and  transfer  systems  provide  air  vehicle  command 
and  control  and  transmit  sensor  data  from  the  air  vehicle  to 
ground  control  stations  for  dissemination  to  fleet  tactical 
operation  centers  and  intelligence  exploitation  sites. 

Mission 

•  Commanders  use  units  equipped  with  MQ-4C  to  conduct 
long  endurance  maritime  surveillance  operations  and  provide 
high-  and  medium-altitude  intelligence  collection. 

-  MQ-4C  operators  will  detect,  identify,  track,  and  assess 
maritime  and  littoral  targets  of  interest  and  collect  imagery 
and  signals  intelligence  information. 

-  Operators  disseminate  sensor  data  to  fleet  units  to  support 
a  wide  range  of  maritime  missions  to  include  surface 
warfare,  intelligence  operations,  strike  warfare,  maritime 
interdiction,  amphibious  warfare,  homeland  defense,  and 
search  and  rescue. 

Major  Contractor 

Northrop  Grumman  Aerospace  Systems,  Battle  Management  and 

Engagement  Systems  Division  -  Rancho  Bernardo,  California 
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Activity 

•  Developmental  flight  testing  of  the  MQ-4C  with  integrated 
mission  systems  and  IFC  2.2  began  in  April  2015.  Poor 
system  stability  and  defect  discovery  consistent  with  early 
developmental  testing  slowed  the  progress  of  developmental 
testing  of  the  mission  systems  and  delayed  the  start  of  the  OA. 

•  The  Navy  began  an  OA  of  the  MQ-4C  in  November  2015,  in 
support  of  a  planned  2QFY16,  Milestone  C  decision.  The  OA 
is  anticipated  to  complete  in  1QFY16.  DOT&E  will  submit  a 
classified  OA  report  in  3QFY16. 

•  The  program  continues  to  pursue  a  solution  providing  traffic 
de-confliction  and  collision  avoidance  capability  since 
development  of  the  Air-to-Air  Radar  Subsystem  was  stopped. 
The  program  intends  to  select  a  technical  solution  in  FY16 
for  delivery  after  IOC.  The  Navy  is  investigating  alternative 
means  of  due  regard  compliance  including  procedures  and 
other  cooperative  avoidance  systems  already  integrated  in  the 
MQ-4C  in  order  to  support  MQ-4C  operations  at  IOC. 

•  Since  the  MQ-4C  is  not  yet  authorized  to  operate  on  Navy 
operational  networks,  the  Navy  did  not  accomplish  a 
cybersecurity  Cooperative  Vulnerability  and  Penetration 
Assessment  (CVPA)  of  the  MQ-4C  during  the  OA. 


Assessment 

•  Traffic  de-confliction  and  collision  avoidance  provide  critical 
mission  capability  for  operation  of  the  MQ-4C  in  civil  and 
international  airspace  in  support  of  global  naval  operations. 
Any  limitation  to  this  capability  at  IOT&E  will  reduce  the 
effectiveness  of  the  MQ-4C. 

•  When  Triton  can  operate  on  operational  networks,  the  Navy 
should  accomplish  a  CVPA  to  allow  the  program  to  determine 
any  cybersecurity  vulnerabilities  in  the  system. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  began  the 
planned  OA  in  FY15  and  intends  to  demonstrate  the  tactics 
and  procedures  necessary  that  will  enable  Triton  to  descend  to 
low  and  medium  altitude. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Demonstrate  any  alternative  means  of  compliance  with  the 
due  regard  requirement  during  developmental  testing  under 
operationally  realistic  conditions  prior  to  IOT&E. 

2.  Conduct  a  CVPA  sufficiently  in  advance  of  the  Adversarial 
Assessment  during  IOT&E  to  allow  the  program  to  correct 
any  discovered  vulnerabilities. 
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Multi-Static  Active  Coherent  (MAC)  System 


Executive  Summary 

•  The  Navy  completed  the  integration  of  the  Multi-Static 
Active  Coherent  (MAC)  Phase  I  system  on  P-8  A  Poseidon 
aircraft  and  conducted  FOT&E  in  accordance  with  a 
DOT&E-approved  test  plan  from  March  2014  through 
February  2015. 

•  P-8A  FOT&E  results  indicate  that  the  MAC  Phase  1  system 
provides  P-8A  aircraft  with  an  early  wide-area  Anti-Submarine 
Warfare  (ASW)  search  capability  in  some  operational 
environments  and  in  select  scenarios,  but  it  does  not  meet  the 
MAC  Phase  I  program’s  requirements  in  other  environments 
or  scenarios.  The  P-8A’s  MAC  wide-area  ASW  search 
capability  is  similar  to  the  capability  on  P-3C  aircraft. 

•  The  FOT&E  did  not  fully  examine  the  capability  of  MAC 
across  all  operational  conditions,  representative  operational 
environments,  and  target  types. 

•  DOT&E  submitted  an  FOT&E  report  on  the  P-8A’s  wide-area 
ASW  search  capability  with  the  MAC  Phase  1  system 

in  December  2015.  The  report  also  updated  DOT&E’s 
assessment  of  the  ASW  mission  capability  provided  by  the 
MAC  system  on  the  P-3C  aircraft. 

•  The  Navy  plans  to  install  MAC  system  software  and  display 
improvements  on  the  P-8A  with  Increment  2  Engineering 
Change  Proposal  (ECP)  2  in  FY15  and  FY16  and  conduct  the 
P-8  A  aircraft  wide-area  ASW  search  assessment  in  mid-FY16. 

System 

•  The  MAC  system  is  an  active  sonar  system  composed  of 
two  types  of  sonobuoys  (source  and  receiver)  and  an  acoustic 
processing  and  aircraft  mission  computer  software  suite.  It  is 
employed  by  the  Navy’s  maritime  patrol  aircraft  (P-3Cs  and 
P-8As)  to  search  for  and  locate  threat  submarines  in  a  variety 
of  ocean  conditions. 

•  MAC  replaces  the  Navy’s  current  Improved  Extended  Echo 
Ranging  (IEER)  system,  which  employs  non-coherent  sources 
to  produce  loud  sounds  that  reflect  off  submarine  targets. 

MAC  employs  new  coherent  source  buoys  that  enable  multiple 
pings,  optimized  waveforms,  and  various  ping  durations,  none 
of  which  the  legacy  IEER  system  provided.  The  Navy  is 
planning  a  series  of  enhancements  to  the  MAC  software  and 
improvements  to  the  MAC  buoys. 


•  To  plan  MAC  missions,  the  Navy  has  updated  the 
Active  System  Performance  Estimate  Computer  Tool 
(ASPECT)/Multi-static  Planning  Acoustics  Toolkit  previously 
used  to  plan  IEER  system  missions. 

•  The  Navy  intends  to  employ  MAC  on  P-3C  and  P-8A  aircraft 
in  a  limited  set  of  acoustic  environments.  Future  increments 
of  MAC  will  be  employed  on  P-8A  aircraft  and  in  a  wider 
variety  of  acoustic  ocean  environments  in  order  to  span  the 
operational  envelope  of  threat  submarine  operations.  MAC 
will  be  the  primary  wide-area  acoustic  search  system  for  the 
P-8A. 

•  MAC  is  expected  to  have  fewer  effects  on  marine  mammals 
and  the  environment  than  the  legacy  IEER  system. 

Mission 

The  Navy  intends  for  P-3C  and  P-8A  crews  equipped  with  MAC 

to  support  the  search,  detect,  and  localization  phases  of  the 

ASW  mission.  MAC  is  particularly  focused  on  large-area  active 

acoustic  searches  for  threat  submarines. 

Major  Contractors 

•  Lockheed  Martin  -  Manassas,  Virginia 

•  Sparton  Electronics  Florida,  Inc.  -  De  Leon  Springs,  Florida 

•  Ultra  Electronics,  Undersea  Sensor  Systems  Incorporated 
(USSI)  -  Columbia  City,  Indiana 

•  Boeing  Defense,  Space,  and  Security  -  St.  Louis,  Missouri 


Activity 

•  The  Navy  completed  integrating  the  MAC  Phase  1  system 
onto  the  P-8A  aircraft  and  conducted  FOT&E  of  the 
P-8A’s  early  wide-area  search  capability  with  the  MAC 
from  March  2014  through  February  2015.  The  Navy 
completed  IOT&E  of  the  MAC  Phase  1  system  on  the  P-3C 


Multi-mission  Aircraft  in  October  2013.  The  Navy  conducted 
the  operational  testing  in  accordance  with  a  DOT&E-approved 
test  plans. 

•  DOT&E  submitted  an  FOT&E  report  on  the  P-8A’s 
wide-area  ASW  search  capability  with  the  MAC  system 
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in  December  2015.  The  report  also  updates  DOT&E’s 
assessment  of  the  ASW  mission  capability  provided  by  the 
MAC  system  on  P-3C  aircraft. 

•  The  Navy  and  DOT&E  commenced  developing  a  Test  and 
Evaluation  Master  Plan  (TEMP),  which  includes  deferred 
MAC  testing  and  plans  for  the  MAC  Phase  2  improvements. 

To  efficiently  utilize  test  resources,  DOT&E  required  the 
test  program  be  consistent  with  and  utilize  MAC  events 
programmed  in  the  approved  P-8A  Increment  2  TEMP. 

Because  future  MAC  testing  is  planned  to  occur  on  the  P-8A, 
the  Navy  intends  to  integrate  the  Phase  2  MAC  test  plan  into 
the  P-8A  Increment  3  TEMP. 

•  The  Navy  and  DOT&E  started  development  of  the  P-8A 
Increment  2  ECP  2  FOT&E  test  plan,  which  allow  the  Navy 
to  evaluate  the  P-8A’s  MAC  system  software  and  display 
improvements  and  complete  deferred  testing  with  diesel 
electric  submarines. 

Assessment 

•  DOT&E  assessed  that  the  MAC  system  provides  the  P-8  A 
and  the  P-3C  aircraft  with  a  wide-area  ASW  search  capability 
in  some  environments  and  for  select  target  scenarios,  but  that 
MAC  falls  short  of  what  the  fleet  identified  as  the  capability 
needed  to  protect  high-value  units.  Although  testing  identified 
P-8A’s  higher  speed  and  revised  buoy  field  installation  tactics 
reduced  the  search  field  buoy  installation  time,  DOT&E 
assessed  MAC  detection  performance  was  similar  to  the 
P-3C’s  performance  and  independent  of  the  aircraft  platform. 
Testing  identified  that  detection  performance  strongly  relies 
on  the  characteristics  of  the  ocean  environment  and  the 
tactics  employed  by  the  target  to  evade  detection.  Testing  to 
understand  the  effects  different  threat  types  and  environments 
have  on  performance  will  continue  through  FY19  in 
conjunction  with  the  P-8A  Increment  2  test  program. 

•  The  P-8A  FOT&E  and  the  P-3C  IOT&E  did  not  fully  examine 
the  capability  of  MAC  across  all  operational  conditions, 
representative  operational  environments,  and  target  types. 
DOT&E  agreed  to  limit  testing  of  the  initial  phase  of  MAC 
because  the  Navy  planned  to  install  MAC  system  software  and 
display  improvements  on  P-8A  during  Increment  2  ECP  2  and 
conduct  additional  testing  to  evaluate  the  P-8A’s  wide-area 
search  capability  in  FY16. 

•  Although  the  MAC  system  demonstrated  a  detection  capability 
against  evasive  undersea  targets,  acoustic  operators  were 
expected  to  quickly  distinguish  system  submarine  detections 
from  a  variety  of  non-submarine  clutter  detections,  some  of 
which  appeared  target-like.  Complicating  this  task,  completed 
test  analysis  identified  that  the  MAC  system  detections 

of  target  and  non-target  clutter  varies  with  environmental 
conditions  and  likely  target  types.  The  data  also  show 
operators  are  only  able  to  recognize  a  small  fraction  of  valid 
system  submarine  detections  as  a  possible  target  and  spent 
time  assessing  and  prosecuting  false  targets. 

•  The  Navy  uses  ASPECT/Multi-static  Planning  Acoustics 
Toolkit  to  develop  MAC  search  plans  and  to  estimate 
theoretical  system  performance.  In  addition  to  the  incomplete 


environmental  databases  used  by  ASPECT  when  modeling 
many  threat  operating  areas,  the  planning  tool  performance 
estimates  are  highly  dependent  on  the  wide-range  of  potential 
mission  planning  input  parameters  estimated  by  the  mission 
planner.  As  a  result,  ASPECT  performance  estimates  can 
widely  vary  when  compared  to  test  results.  The  Navy’s 
Oceanographic  Office  is  updating  these  environmental 
databases,  focusing  first  on  forward  operating  areas.  Since 
ASPECT  does  not  have  a  good  estimate  for  the  operator 
recognition  of  the  submarine  target,  it  overestimates  ASW 
detection  performance. 

•  Additional  information  is  detailed  in  DOT&E’s  classified 
FOT&E  report  on  the  P-8  A  with  MAC  System  dated 
December  2015,  and  the  IOT&E  report  on  the  MAC  System 
on  P-3C  aircraft  dated  July  2014. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  making 
progress  on  the  FY13  and  FY 14  recommendations.  The 
status  of  significant  unclassified  recommendations  remaining 
include: 

1.  The  Navy  Program  Office  is  investigating  fleet  exercise 
data  to  assess  detection  performance  and  to  gather  data  for 
developing  future  algorithm  and  software  improvements. 
The  fleet  exercise  data  include  new  environments  where 
the  fleet  operates  in  peacetime.  The  Navy  is  planning  the 
outstanding  MAC  operational  testing  against  different 
target  types.  This  testing  will  be  in  conjunction  with  P-8A 
Increment  2  ECP-2  FOT&E. 

2.  The  Navy  has  not  completed  development  of  a  sustainable 
MAC  training  program  or  completed  the  formal  updates  to 
tactics  guidelines  and  documentation. 

3.  The  Navy  has  not  developed  processes  for  aircrews  to  better 
understand  the  environmental  conditions  in  the  search  area. 
The  Navy  developed  a  complex  process  for  estimating 

the  environmental  conditions  in  operational  test  areas  to 
improve  ASPECT  predictions;  however,  the  process  is  not 
timely  or  usable  by  typical  aircrews.  The  Navy  should 
continue  to  investigate  methods  for  aircrews  to  measure 
environmental  conditions  in  the  search  area  and  to  adjust 
the  MAC  search  plan  appropriately  when  the  conditions 
change. 

4.  The  Navy  is  investigating  the  MAC  system’s  capability 
for  longer-range  detections  based  on  the  environmental 
conditions  in  the  search  area. 

•  FY  15  Recommendations.  The  Navy  should  implement 
the  recommendations  found  in  DOT&E’s  P-8A  with 
the  MAC  system  FOT&E  report.  DOT&E  provided 

14  recommendations  to  improve  the  MAC  system  performance 
and  6  recommendations  to  improve  test  realism,  minimize 
test  limitations,  and  improve  data  collection.  Significant 
unclassified  recommendations  include: 

1.  Investigate  and  develop  tactics  to  improve  the  operator’s 
ability  to  transition  system  detections  to  high  confidence 
target  detection.  Consider  measures  to  balance  operator 
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workload  and  update  search  plans  based  on  the  actual  3.  Investigate  and  develop  improvements  to  the  ASPECT 


conditions  experienced  in  the  search  area. 

2.  Investigate  the  system’s  capability  for  longer-range 
detections  based  on  the  environmental  conditions  in  the 
search  area. 

planning  system  and  the  supporting  databases. 
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MV-22  Osprey 


Executive  Summary 

•  The  Navy  conducted  OT-IIIK  FOT&E  from  March  2015 
through  August  2015  to  evaluate  the  Mission  Computer 
Obsolescence  Initiative  (MCOI)  and  upgrades  to  the  Defensive 
Weapon  System  (DWS),  the  Ramp  Mounted  Weapon  System 
(RMWS),  and  the  Blue  Force  Tracker  (BFT). 

•  Units  equipped  with  the  modified  MCOI  aircraft  remain 
effective  and  suitable. 

•  The  DWS  was  not  effective  because  of  safety  restrictions  that 
limit  the  use  of  the  DWS  during  arrival  or  departure  from 
landing  zones  and  was  not  suitable  due  to  poor  reliability. 

•  RMWS  effectiveness  was  limited  by  poor  reliability  during  the 
operational  test. 

•  BFT-2  is  not  mature.  Text  messages  were  slow  and  image 
transfers  were  unsuccessful. 

•  The  MV-22  and  supporting  systems  are  vulnerable  to  nearsider 
and  insider  cybersecurity  attacks. 

•  MV-22  OT-IIIK  aircraft  demonstrated  reliability,  availability, 
and  maintainability  consistent  with  the  fleet. 

System 

•  There  are  two  variants  of  the  V-22  Osprey:  the  Marine  Corps 
MV-22  and  the  Air  Force/U.S.  Special  Operations  Command 
CV-22.  The  air  vehicles  for  Air  Force  and  Marine  Corps 
missions  are  nearly  identical,  with  common  subsystems  and 
military  components  sustainable  by  each  Service’s  logistics 
system. 

•  The  Marine  Corps  is  replacing  the  now-retired  CH-46  and 
CH-53D  helicopters  with  the  MV-22.  The  MV-22  is  a  tilt 
rotor  aircraft  capable  of  conventional  wing-borne  flight  and 
vertical  take-off  and  landing. 

•  The  MV-22  can  carry  24  combat-equipped  Marines  and 
operate  from  ship  or  shore.  It  can  carry  an  external  load  up 
to  10,000  pounds  over  50  nautical  miles  and  can  self-deploy 
2,363  nautical  miles  with  a  single  aerial  refueling. 

•  Recent  system  upgrades  include  the  following: 

-  The  MCOI  computer  hardware  initiative  designed  to 
improve  the  performance  of  the  existing  Advanced  Mission 
Computer  architecture  by  adding  greater  processing 
speed  and  more  data  storage  while  maintaining  the  same 
functionality  as  the  original  computer. 

-  GAU-17  DWS  improvements  (upgraded  with  a 
sensor-only  mode  that  allows  the  gunner  to  use  the 


electro-optical  sensor  when  the  gun  turret  is  not  being 
used).  The  DWS  is  a  turreted,  remotely  operated, 
all-quadrant  7.62  mm  defensive  weapon  system  designed 
for  fire  suppression  against  ground  troops  and  soft  targets. 

-  Updated  BFT-2  GPS-enabled  system  that  receives 
infonnation  on  friendly,  neutral,  and  hostile  forces,  as 
well  as  sends  and  receives  text  and  image  messages  via  a 
federated  cockpit  display. 

-  GAU-21  .50  caliber  RMWS  replaced  the  GAU-18  RMWS. 

Mission 

•  Squadrons  equipped  with  MV-22s  will  provide  medium-lift 

assault  support  in  the  following  operations: 

-  Ship-to-Objective  Maneuver 

-  Sustained  operations  ashore 

-  Tactical  recovery  of  aircraft  and  personnel 

-  Self-deployment 

-  Amphibious  evacuation 

Major  Contractors 

Bell-Boeing  Joint  Venture: 

•  Bell  Helicopter  -  Amarillo,  Texas 

•  The  Boeing  Company  -  Ridley  Township,  Pennsylvania 


Activity 

•  The  Navy  conducted  OT-IIIK  FOT&E  from  March  2015 
through  August  2015  in  accordance  with  the 
DOT&E-approved  test  plan  dated  February  20,  2015.  This 


dedicated  OT&E  accumulated  over  130  flight  hours,  which 
had  been  preceded  by  over  400  flight  hours  of  integrated 
testing. 
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•  Marine  Corps  pilots  conducted  testing  at  or  near  the 
Marine  Corps  Air  Station  New  River,  North  Carolina; 

Kirtland  AFB,  New  Mexico;  and  Marine  Corps  Air  Station 
Yuma,  Arizona,  using  three  production-representative 
Advanced  Mission  Computer  aircraft  (which  is  the  original 
pre-MCOI  configuration)  and  one  production  representative 
MCOI  aircraft. 

•  The  Navy  conducted  cybersecurity  testing  of  the  MV-22 
and  its  supporting  systems  at  New  River,  North  Carolina, 
in  May  2015.  DOT&E  observed  all  operational  and 
cybersecurity  testing. 

•  DOT&E  evaluated  capabilities  of  the  latest  aircraft  upgrades  to 
include  the  MCOI,  and  upgrades  to  the  DWS,  RMWS,  and  the 
updated  BFT-2. 

Assessment 

•  Units  equipped  with  MCOI  aircraft  remain  effective  and 
suitable.  During  the  test,  the  unit  successfully  completed  all 
assigned  missions,  but  discovered  deficiencies  that  hindered, 
but  did  not  prevent,  the  ability  of  the  unit  to  perform  its 
mission. 

-  MCOI  aircraft  are  not  compatible  with  the  DWS.  The 
program  manager  has  identified  a  fix  to  this  deficiency  that 
will  be  implemented  and  tested  in  FY 1 6. 

-  The  DWS  was  not  effective  because  of  safety  restrictions 
that  limit  the  use  of  the  DWS  during  arrival  or  departure 
from  landing  zones,  and  was  not  suitable  due  to  poor 
reliability  demonstrated  during  the  operational  test. 

-  The  presence  of  the  DWS  sensor  video  in  the  cockpit 
improved  target  coordination  between  the  gunner  and  the 
pilot.  Pilots  could  now  confirm  the  gunner  was  aiming  at 
the  correct  target  using  the  DWS  cockpit  video,  something 
pilots  previously  could  not  confirm  and  which  resulted 

in  instances  where  the  gunner  mistakenly  shot  the  wrong 
target. 

-  RMWS  effectiveness  was  limited  by  poor  reliability 
during  the  operational  test.  The  mean  rounds  between  an 


uncommanded  stoppage  is  201,  well  below  the  requirement 
of  600  rounds. 

-  BFT-2  image  message  capability  is  not  yet  mature  and 
text  messages  can  be  slow  and  irrelevant  by  the  time 
messages  arrive  at  the  intended  recipient.  No  images  were 
successfully  received  during  operational  testing.  Pilot 
surveys  indicated  some  transmitted  text  messages  were  not 
received 

•  The  MV-22  and  supporting  systems  are  vulnerable  to  nearsider 
and  insider  cybersecurity  attacks. 

•  The  OT-IIIK  MV-22  aircraft  met  reliability  requirements 
and  did  not  meet  maintainability  and  availability  thresholds. 
Demonstrated  reliability,  maintainability,  and  availability 
performance  is  consistent  with  that  of  the  MV-22  fleet. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
progress  addressing  the  previous  FY14  recommendations 
to  continue  to  execute  its  reliability  growth  program  for  the 
MV-22  fleet;  focus  on  reliability  issues  with  the  greatest  effect 
on  operational  availability  and  operational  cost;  and  improve 
the  icing  protecting  system,  which  will  be  tested  in  FY16. 

•  FY  15  Recommendations.  The  Navy  should: 

1.  Address  failure  modes  and  supply  issues  that  limit  aircraft 
availability. 

2.  Use  Marine  Air-Ground  Task  Forces  to  employ  tactics, 
techniques,  and  procedures  to  compensate  for  limitations  in 
the  DWS. 

3.  Integrate  DWS  with  MCOI  and  test  its  capability  in  an 
operational  test. 

4.  Improve  BFT-2  message  latency. 

5.  Investigate  and  improve  RMWS  reliability. 

6.  Address  cyber  vulnerabilities  of  the  MV-22  and  its 
supporting  systems. 
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P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (MMA) 


Executive  Summary 

•  The  Navy  conducted  P-8A  Increment  2  Engineering 
Change  Proposal  (ECP  1)  FOT&E  with  the  Multi-Static 
Acoustic  (MAC)  Phase  1  sensor  system  from  March  2014 
through  February  2015.  The  December  2015  DOT&E  P-8A 
Increment  2  ECP  1  FOT&E  report  concludes  that  the  MAC 
Phase  1  sensor  system  provides  an  early  P-8A  wide-area, 
Anti-Submarine  Warfare  (ASW)  search  capability  similar 

to  P-3C  MAC  search  capability.  DOT&E  assessed  that  the 
P-8A’s  detection  capability  with  the  MAC  Phase  1  sensor 
system  is  strongly  dependent  on  the  environmental  conditions 
present  in  the  search  area  and  the  actions  taken  by  adversaries 
to  avoid  detection.  Although  the  MAC  Phase  1  sensor  system 
provides  an  effective  capability  in  some  environments  and 
scenarios,  it  fails  to  deliver  the  full  capability  described  by  the 
Navy  P-8A  ASW  concept  of  operations  and  MAC  operational 
requirement  documents. 

•  The  Navy  completed  a  P-8A  Data  Storage  Architecture 
Upgrade  (DSAU)  Verification  of  Correction  of  Deficiencies 
FOT&E  event  in  November  2015.  This  test  includes 
verification  of  corrective  actions  implemented  for  nine 
previously  identified  system  deficiencies  and  includes 
evaluation  of  mission  performance  and  cybersecurity  after 
installation  of  an  improved  mission  data  storage  architecture. 

•  The  Navy  continues  to  develop  and  test  corrective  actions 
for  more  than  60  open  system  deficiencies  identified  as 
operationally  significant  during  previous  test  periods, 
including  the  2012  P-8  A  Increment  1  IOT&E  and  2013  P-8A 
Increment  1  FOT&E  events.  The  next  significant  evaluation 
of  deficiency  corrections  is  planned  to  occur  as  early  as  the 
FY16  P-8A  Increment  2  ECP  2  operational  test. 

•  The  Navy  completed  the  second  lifetime  of  the  P-8A  full-scale 
structural  fatigue  testing  in  June  2015.  This  phase  of  testing 
identified  localized  fatigue  cracking  in  non-critical  structural 
components,  including  replaceable  pressure  web  and  aircraft 
tail  section  components.  Preliminary  results  have  not 
identified  any  significant,  near-term  structural  concerns  or  fleet 
operating  limitations.  The  program  is  currently  reviewing 
these  results  to  identify  fleet  airframe  inspection  requirements 
and  depot  repair  procedures  to  ensure  the  airframe  meets  the 
intended  25-year  design  life. 

System 

•  The  P-8A  Poseidon  Multi-mission  Maritime  Aircraft  design 
is  based  on  the  Boeing  737-800  aircraft  with  significant 
modifications  to  support  Navy  maritime  patrol  mission 
requirements.  It  is  replacing  the  P-3C  Orion. 

•  The  P-8A  incorporates  an  integrated  sensor  suite  that  includes 
radar,  electro-optical,  and  electronic  signal  detection  sensors  to 
detect,  identify,  locate,  and  track  surface  targets.  An  integrated 
acoustic  sonobuoy  launch  and  monitoring  system  detects, 


identifies,  locates,  and  tracks  submarine  targets.  Sensor 
systems  also  provide  tactical  situational  awareness  information 
for  dissemination  to  fleet  forces  and  Intelligence,  Surveillance, 
and  Reconnaissance  (ISR)  information  for  exploitation  by  the 
joint  intelligence  community. 

•  The  P-8A  carries  MK  54  torpedoes  and  the  AGM-84D 
Block  1C  Harpoon  anti-ship  missile  system  to  engage 
submarine  and  maritime  surface  targets. 

•  The  P-8A  aircraft  incorporates  aircraft  survivability 
enhancement  and  vulnerability  reduction  systems.  An 
integrated  infrared  missile  detection  system,  flare  dispenser, 
and  directed  infrared  countermeasure  system  is  designed  to 
improve  survivability  against  infrared  missile  threats.  On  and 
off-board  sensors  and  datalink  systems  are  used  to  improve 
tactical  situational  awareness  of  expected  threat  systems. 

Fuel  tank  inerting  and  fire  protection  systems  reduce  aircraft 
vulnerability. 

•  The  Navy  is  integrating  the  MAC  sensor  system  into  the  P-8A 
to  provide  a  wide-area  active,  ASW  search  capability. 

•  Planned  future  upgrades  include  the  addition  of  net-enabled 
ASW  and  ASW  weapons,  high-altitude  ASW  capabilities, 
MAC  wide-area  ASW  search  enhancements,  signals 
intelligence  sensors,  and  advanced  mission  system 
architectures  and  processing  upgrades. 

Mission 

•  Theater  Commanders  primarily  use  units  equipped  with 
the  P-8A  Multi-mission  Maritime  Aircraft  to  conduct  ASW 
operations  including  the  detection,  identification,  tracking,  and 
destruction  of  submarine  targets. 

•  Additional  P-8A  maritime  patrol  missions  include: 

-  ASW  operations  to  detect,  identify,  track,  and  destroy 
enemy  surface  combatants  or  other  maritime  targets 
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ISR  operations  to  collect  and  disseminate  imagery  and 
signals  information  for  exploitation  by  the  joint  intelligence 
community 

C3  operations  to  collect  and  disseminate  tactical  situation 
information  to  fleet  forces 


-  Identification  and  precise  geolocation  of  targets  ashore  to 
support  fleet  Strike  Warfare  missions 

Major  Contractor 

Boeing  Defense,  Space,  and  Security  -  St.  Louis,  Missouri 


Activity 

•  The  Navy  completed  the  MAC  Phase  1  IOT&E  on  the  P-3C 
aircraft  in  2013,  followed  by  integration  of  the  MAC  Phase  1 
sensor  system  into  the  P-8A  aircraft  in  November  2013. 

•  From  March  2014  through  February  2015,  the  Navy  conducted 
the  P-8  A  Increment  2  ECP  1  FOT&E  with  the  MAC  Phase 

1  sensor  system.  During  FOT&E,  the  Navy  conducted 
10  wide-area,  ASW  search  events  to  obtain  the  8  valid  search 
events  required  by  the  DOT&E-approved  test  plan.  Because 
MAC  Phase  1  sensor  system  performance  was  similar  to  P-3C 
performance,  DOT&E  utilized  both  the  P-3C  MAC  IOT&E 
data  and  the  P-8A  MAC  FOT&E  data  to  provide  a  statistically 
significant  assessment  of  P-8A  early  wide-area,  ASW  search 
performance.  DOT&E  submitted  the  P-8  A  Increment  1  ECP 
1  FOT&E  classified  report  for  the  MAC  Phase  I  capability  in 
December  2015. 

•  The  Navy  completed  a  P-8A  DSAU  Verification  of  Correction 
of  Deficiencies  FOT&E  event  in  November  2015.  This 

test  evaluated  improvements  in  ASW  and  ISR  mission  data 
loading  and  storage  following  the  DSAU  modification.  This 
test  event  also  included  testing  to  verify  corrections  for 
nine  previously  identified  weapons  bay,  electronic  signal 
collection,  Information  Assurance,  and  avionics  integration 
deficiencies,  as  well  as  a  system  level  cybersecurity 
assessment. 

•  The  Navy  continues  to  develop  and  test  corrective  actions 
for  more  than  60  open  system  deficiencies  identified  as 
operationally  significant  during  previous  test  periods, 
including  the  2012  P-8A  Increment  1  IOT&E  and  2013  P-8A 
Increment  1  FOT&E  events.  Operational  testing  of  these 
improvements,  including  a  complete  re-evaluation  of  P-8A 
ISR  mission  capabilities,  and  re-evaluation  of  the  P-8A’s  MAC 
wide-area,  ASW  search  capability  is  planned  as  early  as  the 
FY 16  P-8A  Increment  2  ECP  2  FOT&E. 

•  The  Navy  continued  P-8A  Increment  3  test  planning  in  FY15. 
This  next  phase  of  P-8  A  capability  enhancements  includes 
significant  system  architecture  changes,  communication 
system  upgrades,  radar  and  electronic  signal  sensor 
upgrades,  and  the  AGM-84  Harpoon  2+  anti-ship  missile. 
Developmental  testing  will  begin  in  FY16  with  operational 
test  events  planned  for  FY18,  FY19,  and  FY22. 

•  In  FY15,  the  Navy  planned  a  series  of  limited,  quick  reaction 
tests  to  field  P-8A  system  modifications  or  additional 
capabilities  requested  by  fleet  units  or  Combatant  Commands. 
Quick  reaction  operational  testing  supported  fielding  of  the 
UNI-PAC  II  Search  and  Rescue  Kit  and  upgraded  Directed 


Infrared  Countermeasure  sensors  to  provide  full  spherical 
video  coverage  around  the  P-8A  aircraft. 

•  The  Navy  completed  the  second  lifetime  of  the  P-8A  full-scale 
fatigue  and  durability  testing  in  FY15  using  the  fifth  P-8  A 
production  aircraft.  Concurrent  “off-aircraft”  fatigue  testing 
of  the  horizontal  stabilizer  completed  the  equivalent  of 

two  design  lifetimes  of  testing.  Residual  strength  testing  on 
the  full-scale  test  article  and  horizontal  stabilizer  will  continue 
into  FY16.  Main  and  nose  landing  gear  subassemblies 
completed  the  equivalent  of  three  lifetimes  of  fatigue  testing  in 
FY  15.  Landing  gear  post-test  teardown  and  data  review  is  in 
progress.  Final  fatigue  test  analysis  and  results  are  expected  to 
be  available  in  late  FY16. 

Assessment 

•  The  2015  DOT&E  P-8A  Increment  2  ECP  1  FOT&E  classified 
report  concludes  that  P-8A  MAC  Phase  1  and  P-3C  MAC 
Phase  1  sensor  systems  provide  similar  wide-area,  ASW  search 
capabilities  that  are  not  currently  provided  by  other  passive 
search  systems.  Test  results  show  that  MAC  Phase  1  sensor 
capabilities  on  both  aircraft  platforms  are  strongly  dependent 
on  search  area  environmental  conditions  and  adversary  evasive 
actions.  During  OT&E,  MAC  Phase  1  clearly  provided  an 
effective  capability  in  some  test  environments  and  target 
evasion  profiles,  but  failed  to  deliver  the  full  capability 
required  by  the  Navy’s  concept  of  operations  and  MAC 
operational  requirement  documents.  Additional  information 
on  MAC  Phase  1  performance  is  contained  in  the  DOT&E 
P-3C  MAC  Phase  1  IOT&E  classified  report  released  in  2014. 

•  The  Navy  is  making  progress  toward  correcting  the  more 
than  60  open  system  deficiencies  identified  as  operationally 
significant  in  previous  test  periods,  including  the  2012  P-8A 
Increment  1  IOT&E  and  2013  P-8A  Increment  1  FOT&E 
events.  Some  of  these  corrections  are  required  to  address 
the  mission  capability  shortfalls  identified  in  previous  P-8A 
operational  test  reports,  including  both  the  Navy  and  DOT&E 
assessments  that  the  fielded  P-8A  Increment  1  system  is  not 
operationally  effective  for  the  ISR  mission.  While  the  series 
of  P-8  A  software  and  hardware  improvements  released  for 
fleet  use  since  2012  have  addressed  some  of  these  problems, 
the  majority  remain  unresolved.  Operational  testing  of  these 
improvements,  including  a  complete  re-evaluation  of  P-8A 
ISR  mission  capabilities,  is  planned  as  early  as  the  FY16  P-8  A 
Increment  2  ECP  2  FOT&E  event. 
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•  The  Navy  completed  the  P-8A  DSAU  Verification  of 
Correction  of  Deficiencies  FOT&E  event  in  November 
2015.  Initial  test  observations  indicate  that  the  DSAU 
modification  provides  improved  mission  data  loading  and 
storage  capabilities.  In  addition,  this  test  demonstrated  at  least 
partial  correction  of  nine  previously  documented  deficiencies 
to  improve  mission  capabilities  and  operator  interfaces. 
Preliminary  cybersecurity  test  observations  identified  some 
system  vulnerabilities  that  could  potentially  be  exploited  to 
create  significant  mission  effects.  Navy  and  DOT&E  data 
review  and  analysis  are  in  progress.  Complete  test  results  will 
be  available  in  2QFY16. 

•  The  Navy  completed  the  second  lifetime  of  the  P-8A  full-scale 
structural  fatigue  testing  in  June  2015.  This  phase  of  testing 
identified  localized  fatigue  cracking  in  non-critical  structural 
components,  including  replaceable  pressure  web  and  aircraft 
tail  section  components.  Preliminary  results  have  not 
identified  any  significant,  near-term  structural  concerns  or  fleet 
operating  limitations.  The  program  is  currently  reviewing 
results  to  identify  initial  fleet  airframe  inspection  requirements 
and  depot  repair  procedures  to  ensure  the  airframe  meets  the 
intended  25-year  design  life. 

•  The  Navy  conducted  a  limited  user  test  in  June  2015,  to  certify 
P-8A  carriage  of  the  self-contained  UNI-PAC  II  Search  and 
Rescue  Kit.  This  payload  provides  delivery  of  a  life  raft  and 
survival  accessories  for  up  to  eight  survivors.  The  Navy  also 
conducted  testing  to  support  replacement  of  current  Directed 
Infrared  Countermeasure  sensors  with  upgraded,  two-color 
infrared  sensors  in  July  2015.  This  test  was  in  response  to 


a  Pacific  Fleet  Urgent  Operational  Need  request  to  provide 
spherical  video  coverage  for  Maritime  Patrol  Reconnaissance 
Aircraft. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  made 
progress  on  all  three  FY14  recommendations.  An  FOT&E 
event  to  verify  correction  of  an  additional  nine  operationally 
significant  system  deficiencies  is  currently  in  progress,  with  a 
more  significant  deficiency  correction  FOT&E  event  planned 
for  FY16.  The  initial  phase  of  P-8A  MAC  wide-area,  ASW 
search  testing  was  completed  in  FY15,  with  additional  testing 
planned  for  FY16.  Test  planning  for  P-8A  Increment  2  and 
Increment  3  high  altitude,  ASW  mission  capabilities  is  also  in 
progress. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Continue  to  implement  corrective  actions  for  all 
operationally  significant  system  deficiencies  identified 
in  previous  P-8A  operational  test  reports  and  conduct 
additional  follow-on  operational  tests  to  verify  improved 
mission  capabilities. 

2.  Continue  to  conduct  planned  P-8A  Increment  2  MAC 
operational  testing  to  evaluate  ASW  mission  capability 
improvements. 

3.  Submit  a  comprehensive  P-8A  Test  and  Evaluation  Master 
Plan  for  DOT&E  approval  that  incorporates  a  test  strategy 
for  previous  operational  deficiency  corrections  and  the 
significant  mission  capability  enhancements  included  in  the 
P-8  A  Increment  3  program. 
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Remote  Minehunting  System  (RMS) 


Executive  Summary 

•  In  FY15,  developmental  and  integrated  testing  of  Remote 
Minehunting  System  (RMS)  upgrades  (consisting  of  a 
version  6.0  (v6.0)  Remote  Multi-Mission  Vehicle  (RMMV) 
and  AN/AQS-20A/B  sonar)  did  not  demonstrate  sufficient 
performance  or  successful  integration  with  interfacing 
Littoral  Combat  Ship  (LCS)  systems  to  achieve  the  Navy’s 
minimum  Increment  1  warfighting  capability.  In  particular, 
testing  of  the  v6.0  RMMV  provides  no  statistical  evidence 
of  reliability  improvement,  and  the  Navy  continues  to 
experience  frequent  problems  with  LCS-based  launch, 
handling,  and  recovery  equipment  and  communications 
systems  essential  for  conducting  timely  and  sustained  Mine 
Countermeasures  (MCM)  operations.  In  addition,  following 
problems  encountered  during  testing  in  December  2014  and 
January  2015,  the  Navy  abandoned  its  earlier  plan  to  develop 
and  field  an  improved  sonar  (designated  AN/AQS-20B)  by  the 
end  of  FY15. 

•  In  an  August  2015  memorandum,  DOT&E  advised  the 
USD(AT&L)  that  the  reliability  of  the  RMS  and  its  RMMV 
poses  a  significant  risk  to  the  planned  operational  test  of  the 
Independence  variant  LCS  and  the  Increment  1  MCM  mission 
package,  and  to  the  Navy’s  plan  to  field  and  sustain  a  viable 
LCS-based  minehunting  and  mine  clearance  capability  prior 
to  FY20.  DOT&E  recommended  that  the  acquisition  strategy 
for  these  systems  be  reexamined  to  ensure  that  sufficient 
testing  is  performed  to  inform  the  procurement  of  additional 
vehicles  and  cautioned  that  continued  development  of  this 
program  without  a  fundamental  change  would  be  unlikely 

to  result  in  a  system  that  is  effective  and  suitable.  Test  data 
collected  throughout  FY15  continue  to  refute  the  Navy’s 
assertion  that  vehicle  reliability  has  improved.  Moreover, 
the  current  estimates  of  RMMV  and  RMS  reliabilities  are 
22.7  and  18.3  hours  Mean  Time  Between  Operational  Mission 
Failure  (MTBOMF),  which  are  well-short  of  what  is  needed 
to  complete  MCM  missions  in  a  timely  fashion  and  meet  the 
Navy’s  desired  mission  timelines. 

•  The  Navy  chartered  an  independent  program  review  of  the 
RMS  including  an  evaluation  of  potential  alternative  MCM 
systems  in  September  2015.  The  independent  review  team’s 
report  is  due  in  late  1QFY16.  Meanwhile,  USD(AT&L) 
delayed  its  review  to  consider  approval  to  restart  RMS 
low-rate  initial  production  until  at  least  3QFY16. 

•  DOT&E  concluded  in  a  November  2015  memorandum  to 
USD(AT&L)  and  the  Navy,  based  on  the  testing  conducted 
to  date,  that  an  LCS  employing  the  current  MCM  mission 
package  would  not  be  operationally  effective  or  operationally 


suitable  if  the  Navy  called  upon  it  to  conduct  MCM  missions 
in  combat.  Five  of  seven  primary  shortcomings  supporting 
this  conclusion  were  attributed,  at  least  in  part,  to  the  RMS: 

-  Critical  MCM  systems,  including  the  RMMV,  are  not 
reliable. 

-  Vulnerabilities  of  the  RMMV  to  mines  and  its  high  rate  of 
failures  do  not  support  sustained  operations  in  potentially 
mined  waters. 

-  RMMV  operational  communications  ranges  are  limited. 

-  Minehunting  capabilities  are  limited  in  other-than-benign 
environmental  conditions. 

-  The  LCS  crew  is  not  equipped  to  maintain  the  ship  or  the 
MCM  systems. 

•  Developmental  and  integrated  testing  conducted  in  FY15 
continued  to  show  that  the  AN/AQS-20A  sonar  does  not  meet 
all  Navy  requirements.  The  RMS  has  not  demonstrated  the 
detection/classification  and  localization  capabilities  needed 
for  an  LCS  equipped  with  an  Increment  1  MCM  mission 
package  to  complete  timely  mine  reconnaissance  and  mine 
clearance  operations  in  expected  operational  environments. 

In  addition,  testing  has  revealed  several  shortcomings  that, 
unless  corrected,  will  delay  completion  of  LCS-based  mine 
reconnaissance  and  mine  clearance  operations.  The  Navy 
expected  to  correct  these  deficiencies  prior  to  operational 
testing  by  implementing  pre-planned  product  improvements 
(the  AN/AQS-20B  version  of  the  sonar)  and  integrating  the 
improved  sensor  into  the  MCM  mission  package.  However, 
system  prototypes  did  not  perform  well  in  initial  testing  in 
FY14/15  and  the  Navy  elected  to  proceed  to  LCS  MCM 
mission  package  Technical  Evaluation  (TECHEVAL)  with  the 
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AN/AQS-20A  sonar  with  known  limitations  outside  the  most 
benign  conditions. 

•  Communications  ranges  afforded  by  current  RMS  radios 
will  require  operational  commanders  to  clear  a  series  of 
LCS  operating  boxes  to  support  minehunting  and  clearance 
operations,  particularly  for  bottom- focused  mine-hunting 
operations.  These  operating  boxes  will  be  necessary  to  keep 
an  LCS  and  its  crew  out  of  the  minefield  while  operating  the 
RMS  in  searches  for  mine-like  objects  or  identifying  bottom 
objects  located  within  shipping  lanes  that  are  longer  than 
demonstrated  communications  ranges.  Additional  effort 

to  clear  operating  boxes  will  increase  the  demand  for  mine 
clearance  and  delay  attainment  of  strategic  objectives.  During 
FY15  testing,  LCS  2  also  had  frequent  problems  establishing 
initial  communications  between  the  ship  and  an  RMMV  using 
existing  over-the-horizon  (OTH)  and  line-of-sight  (LOS) 
channels  and  maintenance  of  those  communications  links 
once  established.  These  problems  frequently  delayed  the 
start  of  RMS  missions  and  periodically  terminated  missions 
prematurely. 

•  Although  the  Navy  implemented  materiel,  training,  and 
procedural  improvements,  incidents  of  equipment  damage 
and  launch  and  recovery  failures  continue  to  delay  or  prevent 
sustained  operations.  The  Navy  completed  16  RMMV 
launches  and  14  RMMV  recoveries  during  23  days  at  sea  in 
developmental  testing  completed  in  1QFY15.  During  58  days 
at  sea  during  TECHEVAL,  both  the  pace  and  success  rate  of 
RMMV  launch  and  recovery  regressed  as  LCS  2  completed 
24  RMMV  launches  and  only  18  RMMV  recoveries.  The 
increased  frequency  of  unrecovered  RMMVs  is  attributed  to 

a  larger  number  of  off-board  vehicle  failures  that  precluded 
recovery  aboard  LCS  2  rather  than  new  launch  and  recovery 
problems.  Damage  to  shipboard  launch  and  recovery 
equipment,  LCS-RMMV  communications  problems,  multiple 
RMMV  hydraulics  system  failures,  a  suspected  RMMV 
electrical  system  failure,  and  RMMV  mast  latch  and  fuel 
system  failures  contributed  to  the  ship’s  inability  to  launch  or 
recover  the  unmanned  vehicle. 

•  The  combination  of  acoustic  radiated  noise,  frequent  RMMV 
failures  that  prevent  recovery  aboard  LCS,  and  the  probability 
the  vehicle  and  its  sensor  will  get  entangled  with  mine  or 
other  hazards  all  pose  a  risk  to  losing  the  RMS.  Given  the 
limited  existing  inventory  of  RMMVs  (four  v6.0  vehicles, 
four  vehicles  awaiting  upgrades  to  v6.0,  and  two  vehicles 
designated  for  training  use  only),  any  RMMV  attrition  would 
severely  degrade  the  Navy’s  ability  to  conduct  LCS-based 
MCM  operations. 


System 

•  The  RMS  is  designed  to  provide  off-board  mine 
reconnaissance  capability  to  detect,  classify,  and  localize 
non-buried  bottom  and  moored  mines,  and  to  identify 
shallow-water  bottom  mines  only. 

•  The  Navy  plans  to  launch,  operate,  and  recover  RMS  from 
both  LCS  Flight  0/0+  variants  as  part  of  the  MCM  mission 
package  (when  embarked). 

•  RMS  includes  an  unmanned,  diesel-powered,  semi 
submersible  vehicle  called  the  RMMV.  The  RMMV  tows  an 
AN/AQS-20  variable  depth  sonar  mine  sensing  subsystem. 

The  AN/AQS-20  is  a  multi-mode,  modular  towed  body  that 
can  house  as  many  as  five  sonars.  The  sensor  can  also  be 
fitted  with  an  electro-optical  identification  device  to  identify 
mine-like  objects.  The  Navy  is  developing  an  improved 
forward-looking  sonar  and  new  synthetic  aperture  side-looking 
sonars  that  it  expects  to  field  in  the  AN/AQS-20B/C  by 
FY18/19.  Following  suspension  of  MH-60S  tow  missions  in 
2011,  the  RMMV  is  currently  the  only  vehicle  that  tows  the 
AN/AQS-20. 

•  A  datalink  subsystem  provides  real-time  communications 
between  the  host  ship  and  the  RMMV  for  command  and 
control  and  transmission  of  some  sensor  data.  The  RMS 
datalink  subsystem,  which  includes  ultra-high  frequency  LOS 
and  low-band  very  high  frequency  OTH  radios,  interfaces  with 
the  multi-vehicle  communications  system  that  resides  on  both 
LCS  variants. 

•  Shipboard  operators  control  the  RMMV  using  a  remote 
minehunting  functional  segment  integrated  into  the  LCS 
mission  package  computing  environment. 

•  RMS  sensor  data  are  recorded  to  a  removable  hard  drive 
during  minehunting  operations.  Following  vehicle  recovery, 
operators  transfer  data  to  an  organic  post  mission  analysis 
station  and  review  sonar  data  to  mark  contacts  as  suspected 
mine-like  objects. 

Mission 

MCM  Commanders  will  employ  the  RMS  from  an  MCM  mission 

package-equipped  LCS,  to  detect,  classify,  and  localize  non 

buried  bottom  and  moored  mines,  and  to  identify  shallow-water 

bottom  mines  in  support  of  theater  minehunting  operations. 

Major  Contractors 

•  RMMV:  Lockheed  Martin  -  West  Palm  Beach,  Florida 

•  AN/AQS-20  (all  variants):  Raytheon 
Corporation  -  Portsmouth,  Rhode  Island 


Activity 

•  During  1QFY15,  the  Navy  completed  the  last  scheduled  phase 
of  the  Increment  1  MCM  mission  package  developmental  test 
DT-B2  aboard  USS  Independence  (LCS  2).  This  phase  of 
testing  marked  the  first  time  the  Navy  employed  v6.0  RMMVs 


in  the  MCM  mission  package.  The  test  also  provided  the  first 
opportunity  to  assess  ship-based  RMS  operations  that  were 
unable  to  be  completed  in  earlier  events,  described  by  the 
RMS  Test  and  Evaluation  Master  Plan  (TEMP),  because  of 
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LCS  unavailability  and  deferred  integration  of  RMMV  and 
LCS. 

•  The  Navy  conducted  shore-based  developmental  testing 
(DT-B1)  of  the  RMS,  consisting  of  the  v6.0  RMMV  and 
AN/AQS-20A/B  from  the  contractor’s  facility  at  West 
Palm  Beach,  Florida.  The  Navy  commenced  testing  in 
December  2014  with  an  upgraded  version  of  the  sensor, 
designated  AN/ AQS-20B,  but  in  January  2015  determined 
the  new  sensor  was  not  yet  sufficiently  mature  and  elected 
to  complete  testing  with  the  AN/AQS-20A  sonar.  The  Navy 
subsequently  suspended  testing  in  January  2015  to  investigate 
RMMV  reliability  problems  and  complete  corrective 
maintenance.  The  Navy  resumed  and  completed  testing  in 
March  2015. 

•  Although  the  Navy  elected  to  proceed  to  LCS  and  Increment  1 
MCM  mission  package  TECHEVAL  with  the  AN/AQS-20A, 
it  continued  to  develop  pre-planned  product  improvements 
for  the  sonar.  This  effort  is  intended  to  mitigate  deficiencies 
observed  during  previous  operational  assessments  and 
developmental  testing  of  the  RMS  and  AN/AQS-20A. 
Although  the  Navy  no  longer  plans  to  field  the  AN/AQS-20B, 
it  will  continue  to  perform  risk  reduction  testing  of  the  v6.0 
RMMV  and  the  improved  AN/AQS-20B  sensor  in  FY16. 

•  From  April  through  August  2015,  the  Navy  employed  four 
v6.0  RMMVs  in  TECHEVAL  of  the  Independence  variant 
LCS  and  Increment  1  MCM  mission  package  aboard  LCS 
2.  Although  the  Navy  planned  to  complete  the  test  by  June 
2015,  problems  with  failures  of  seaframe  and  MCM  systems, 
including  RMMVs,  caused  the  testing  to  be  extended.  The 
Navy  delayed  operational  testing  of  the  Increment  1  MCM 
mission  package,  which  it  expected  to  complete  in  FY15,  until 
the  spring  of  2016,  at  the  earliest. 

•  In  June  2015,  the  Navy  commenced  RMS  cybersecurity 
operational  testing  concurrently  with  LCS  2  cybersecurity 
testing.  The  initial  phase  of  the  cybersecurity  operational 
test,  a  Cooperative  Vulnerability  and  Penetration  Assessment 
was  completed  in  July  2015,  but  the  final  phase  of  the  test,  an 
Adversarial  Assessment,  is  on  hold  pending  a  Navy  decision 
on  the  readiness  of  the  Increment  1  MCM  mission  package 
and  Independence  variant  seaframe  for  operational  testing. 

•  In  an  August  2015  memorandum,  DOT&E  advised  the 
USD(AT&L)  that  the  reliability  of  the  RMS  and  its  RMMV 
is  so  poor  that  it  poses  a  significant  risk  to  the  planned 
operational  test  of  the  Independence  variant  LCS  and  the 
Increment  1  MCM  mission  package,  and  to  the  Navy’s  plan 
to  field  and  sustain  a  viable  LCS-based  minehunting  and  mine 
clearance  capability  prior  to  FY20.  DOT&E  recommended 
that  the  acquisition  strategy  for  these  systems  be  reexamined 
to  ensure  that  sufficient  testing  is  performed  to  inform 

the  procurement  of  additional  vehicles  and  cautioned  that 
continued  development  of  this  program  without  a  fundamental 


change  would  be  unlikely  to  result  in  a  system  that  is  effective 
and  suitable. 

•  The  Navy  chartered  an  independent  program  review  of  the 
RMS,  including  an  evaluation  of  potential  alternative  MCM 
systems  in  September  2015.  Their  report  is  due  in  late 
1QFY16.  Additionally,  USD(AT&L)  delayed  its  review  to 
consider  approval  to  restart  RMS  low-rate  initial  production 
was  delayed  until  at  least  3QFY16. 

•  In  November  2015,  DOT&E  provided  the  USD(AT&L),  the 
Assistant  Secretary  of  the  Navy  for  Research  Development  and 
Acquisition,  and  the  Program  Executive  Officer  for  Littoral 
Combat  Ships  a  classified  assessment  of  the  performance  of 
the  Independence  variant  seaframe  and  Increment  1  MCM 
mission  package,  including  the  RMS.  DOT&E  based  the 
assessment  on  the  data  collected  during  the  TECHEVAL  and 
earlier  periods  of  development  and  operational  testing. 

•  In  FY15,  the  Navy  continued  an  effort,  initiated  in  2QFY14,  to 
update  the  RMS  and  AN/AQS-20A  TEMPs.  DOT&E  advised 
the  Navy  that  both  TEMPs  should  be  further  combined  in  the 
LCS  TEMP,  which  is  also  being  updated.  It  remains  unclear 
when  the  Navy  will  complete  updates  to  either  the  RMS  or 
LCS  TEMPs. 

Assessment 

•  DOT&E ’s  assessment  is  based  on  information  from 
developmental  and  integrated  testing,  results  provided  by  the 
Navy  Program  Offices,  operational  assessments  of  the  RMS 
and  AN/AQS-20A,  and  operational  cybersecurity  testing 
aboard  LCS  2.  A  summary  of  the  RMS  portion  of  DOT&E’s 
recent  memorandum  on  LCS  2  and  Increment  1  MCM  mission 
package  TECHEVAL  is  also  provided  below. 

Reliability  Growth 

•  The  RMS  program,  which  the  Navy  initiated  in  1993, 
has  a  history  of  reliability  problems.  The  Navy  instituted 
reliability  improvement  initiatives  when  the  v4.0  system  did 
not  meet  its  reliability  or  availability  requirements  during 
an  aborted  IOT&E  in  2007.  Following  an  operational 
assessment  in  2008  and  a  Nunn-McCurdy  review  of 
the  program  in  2010,  the  Navy  assessed  v4.0  system 
reliability  as  44.4  hours  MTBOMF  when  it  embarked  on 
a  three-phased  reliability  growth  program  (v4.1,  v4.2,  and 
v4.3)  designed  to  retire  RMMV  failure  modes  and  improve 
reliability.  DOT&E  assessed  v4.0  RMMV  reliability 
as  21.8  hours  MTBOMF  based  on  developmental  and 
operational  testing  completed  prior  to  June  2010.  The 
figure  below  provides  a  comparison  of  Navy  and  DOT&E 
reliability  assessments  of  v4.0  RMMV  and  subsequent 
vehicle  configurations  employed  in  developmental  and 
integrated  testing.  The  Navy  assessed  v4.2  RMMV 
reliability  twice  obtaining  two  different  results. 
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•  By  June  2013,  the  Navy  indicated  it  had  grown  reliability  of 
the  v4.2  RMMV  configuration  to  213.7  hours,  and  declared 
that  the  third  phase  of  its  reliability  growth  program  (v4.3) 
was  no  longer  necessary.  Following  developmental  and 
integrated  testing  that  fall,  the  Navy  reported  v4.2  RMMV 
reliability  was  192.5  hours  MTBOMF.  Based  on  the  same 
testing,  DOT&E  assessed  that  RMMV-only  reliability  was 
31.3  hours  MTBOMF  when  counting  only  failures  that  would 
have  required  intermediate-  or  depot-level  intervention  to 
fully  correct.  In  addition,  DOT&E’s  assessment  noted  that 
the  Navy  had  inflated  operating  time  estimates  in  its  reliability 
calculation  by  assuming  post-mission  analysis  time  (when 
the  vehicle  is  not  in  the  water  and  not  operating)  could  be 
counted.  DOT&E  also  assessed  that  the  Navy’s  calculation 
missed  several  critical  failures  that  precluded  continuation  of 
operational  missions.  DOT&E’s  assessment  of  v4.2  RMMV 
reliability  also  identified  specific  deficiencies  the  Navy 

had  not  yet  corrected  in  two  phases  of  its  reliability  growth 
program  but  hoped  to  mitigate  in  conjunction  with  vehicle 
upgrades  (v6.0)  required  to  make  it  more  compatible  with 
LCS  communications  and  launch,  handling,  and  recovery 
systems.  Although  the  Navy  acknowledged  these  deficiencies, 
they  determined  that  v4.2  RMMV  testing  had  demonstrated 
that  it  met  its  Nunn-McCurdy  exit  criterion  for  reliability  in 
preparation  for  a  potential  Milestone  C  decision  and  restart  of 
low-rate  initial  production. 

•  In  an  August  2015  memorandum  to  USD(AT&L),  DOT&E 
assessed  that  the  v6.0  system  the  Navy  is  relying  upon  to 


underpin  the  first  increment  of  the  LCS  MCM  mission 
package  continued  to  exhibit  reliability  problems  in  both 
shore-  and  LCS-based  testing.  In  the  same  memorandum, 
DOT&E  assessed  that  recent  developmental  testing  provided 
no  statistical  evidence  that  the  system  was  demonstrating 
improved  reliability,  and  instead  indicated  that  reliability 
plateaued  nearly  a  decade  ago.  The  figure  above  shows 
DOT&E  and  the  Navy  reporting  comparable  quantitative 
results  for  v6.0  RMMV  reliability  based  on  partial 
TECHEVAL  data  available  at  that  time.  Moreover,  the  Navy 
assessment  of  v6.0  RMMV  reliability,  39  hours  MTBOMF, 
provides  evidence  supporting  DOT&E’s  conclusion  that 
reliability  has  not  improved  despite  multiple  upgrade  phases 
since  the  program  exited  its  Nunn-McCurdy  review  in  2010. 
The  figure  also  shows  that  the  Navy’s  estimate  for  v6.0 
RMMV  reliability  is  still  less  than  the  Navy’s  estimate  of 
v4.0  RMMV  reliability  at  the  outset  of  its  reliability  growth 
program. 

•  In  total,  RMS  operated  for  265.7  hours  between  April  7,  2015, 
when  LCS  2  began  scenario-based  MCM  workups,  and 
August  30,  2015,  when  TECHEVAL  concluded.  During  this 
test  period,  the  RMS  experienced  17  operational  mission 
failures  with  15  of  those  failures  attributable  to  the  RMMV. 
Thus,  as  shown  in  the  table  below,  the  reliabilities  of  RMS 
and  v6.0  RMMV  were  15.6  hours  and  17.7  hours  MTBOMF, 
respectively,  during  TECHEVAL.  When  TECHEVAL  data 
are  combined  with  previous  data,  reliabilities  RMS  and  v6.0 
RMMV  are  18.3  hours  and  22.7  hours  MTBOMF. 


RMS  and  v6.0  RMMV  Reliability  in  2014-201 5  Testing 


Test  Event 

Test  Period 

System  Operating 
Time  (Hours) 

RMMVOMFs 

RMMV  MTBOMF 
(Hours) 

RMSOMFs 

RMS  MTBOMF 
(Hours) 

LCS  MCM  MP 
DT-B2  Ph4  Pd 2 

Sept  11  -  Oct  20, 2014 

139.0 

3 

46.3 

(20.8-126.1) 

6 

23.2 

(13.2-44. 1) 

DT-B1 

Jan  13 -Mar  25, 2015 

163.4 

7 

23.3 

(13.9-42.0) 

8 

20.4 

(12.6-35. 1) 

LCS  MCM  MP 
TECHEVAL 

Apr  7 -Aug  30,2015 

265.7 

15 

111 

(12.5-25.8) 

17 

15.6 

(1 1.3-22.2) 

All 

Sep  11, 201 4 -Aug  30, 201 5 

568.1 

25 

22.7 

(17.4-30.1) 

31 

18.3 

(14.4-23.6) 

Note:  Values  in  parentheses  represent  80  percent  confidence  intervals. 

MCM  -  Mine  Countermeasures;  MP  -  mission  package;  TECHEVAL  -  Technical  Evaluation;  RMMV  -  Remote  Muti-Mission  Vehicle;  OMF  -  Operational  Mission  Failure; 

MTBOMF  -  Mean  Time  Between  Operational  Mission  Failure 
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•  As  DOT&E  assessed  in  August  2015,  the  reliability  of 
existing  systems  poses  a  significant  risk  to  the  Navy’s  plan  to 
field  and  sustain  a  viable  LCS-based  minehunting  and  mine 
clearance  capability  prior  to  FY20.  In  particular,  recurrent 
failures  preventing  vehicle  recovery  aboard  LCS,  problems 
establishing  and  maintaining  RMMV-LCS  communications, 
the  accelerated  failure  of  control  surface  actuators,  and  the 
need  for  frequent  intermediate-  and  depot-level  assistance  to 
initiate  and  continue  sorties  continue  to  handicap  the  crew’s 
ability  to  sustain  system  operations.  Unless  corrected,  these 
problems  will  continue  to  prevent  the  Navy  from  fielding  an 
LCS  and  MCM  mission  package  capable  of  replacing  legacy 
systems  or  decreasing  significantly  the  time  required  to 
conduct  MCM  operations. 

•  While  the  Navy  agrees  that  existing  RMMVs  fail  at  a  high 
rate  and  are  demonstrating  reliability  that  is  less  than  required, 
it  believes,  the  system  and  the  MCM  mission  package  can 
still  accomplish  their  intended  missions.  This  is  incorrect. 
TECHEVAL  provided  numerous  examples  of  system 
shortcomings  that  prevented  the  Navy  from  demonstrating 
RMS  operating  tempo  over  an  extended  period  of  time  that 
was  close  to  the  expectations  of  the  Navy’s  Design  Reference 
Mission  Profile  for  the  LCS  equipped  with  the  MCM  mission 
package. 

-  During  TECHEVAL,  four  RMMVs  and  six  AN/AQS-20As 
operated  off-board  LCS  for  226  hours  and  conducted 

94  hours  of  minehunting  (employing  the  sonar  to  actively 
search  for  mines,  revisit  contacts,  and  identify  bottom 
objects).  On  six  occasions,  an  RMMV  could  not  be 
recovered  aboard  LCS  2  and  had  to  be  towed  to  port  by 
test  support  craft,  and  then  shipped  to  the  remote  operating 
site  (simulating  an  in-theater  intermediate-  and  depot-level 
maintenance  activity)  or  prime  contractor  site  (original 
equipment  manufacturer  depot-level  repair  facility)  for 
repairs.  On  average,  the  LCS  2  completed  a  total  of 
5  hours  of  RMS  minehunting  per  week  (1.25  hours  per 
week  per  RMMV),  and  an  RMMV  had  to  be  towed  to  port 
for  every  16  hours  of  RMS  minehunting. 

-  The  pace  of  RMS  operations  demonstrated  by  one  LCS 
with  four  RMMVs  is  less  than  10  percent  of  the  operating 
tempo  for  a  single  ship  shown  in  the  Navy’s  Design 
Reference  Mission  Profile  for  Increment  1  bottom-focused 
minehunting  operations.  Based  on  the  demonstrated  pace 
of  operations  during  TECHEVAL,  all  of  the  RMMVs 

the  Navy  plans  to  acquire  to  outfit  24  MCM  mission 
packages  would  be  required  to  search  the  area  that  the 
Navy  originally  projected  a  single  LCS  and  MCM  mission 
package  could  search. 

-  Although  the  Navy  considers  one  of  the  two  RMMVs  in 
the  Increment  1  mission  package  an  embarked  spare  that 
permits  continued  RMS  operations  even  after  one  unit 
fails,  LCS  2  averaged  just  3.5  days  underway  before  losing 
all  RMS  capability,  that  required  a  call  for  outside  RMS 
repair  assistance,  or  necessitated  a  return  to  port. 


-  LCS  2  was  underway  for  more  than  one  week  with  at  least 
one  mission-capable  RMS  embarked  only  once  during 
TECHEVAL. 

-  On  five  occasions,  LCS  2  operated  for  less  than 
two  days  before  encountering  an  RMS  problem  that 
required  assistance  from  shore-based  intermediate-level 
maintenance  personnel  to  continue  operations. 

-  In  three  cases,  an  RMMV  was  recovered  without  collecting 
minehunting  data.  These  problems  resulted  in  the 
RMMV  returning  to  LCS  2  with  at  least  some  fraction 

of  the  expected  mission  data  in  only  15  of  24  launches 
(63  percent). 

-  Mishaps  severely  damaged  two  RMMVs,  causing  them 
to  be  returned  to  the  prime  contractor’s  site  for  extensive 
repairs. 

-  Despite  underway  periods  that  were  short  relative  to  the 
expectations  of  the  LCS  Design  Reference  Mission  Profile, 
both  RMMVs  embarked  at  the  beginning  of  an  underway 
period  were  unavailable  to  conduct  minehunting  missions 
six  times  during  TECHEVAL. 

-  On  three  occasions,  totaling  19  days,  all  four  v6.0  RMMVs 
in  the  Navy’s  inventory  were  unavailable  to  execute 
minehunting  missions. 

-  The  Navy  completed  TECHEVAL  with  one  of  four 
RMMVs  operational.  However,  post-test  inspections 
revealed  that  the  sonar  tow  cable  installed  in  that  unit  was 
no  longer  functional. 

Minehunting  Performance 

•  The  RMS  program  has  not  yet  demonstrated  that  the 
AN/AQS-20A  can  meet  its  detection  and  classification 
requirements  over  the  prescribed  depth  regimes  and 
simultaneously  provide  adequate  coverage  against  all 
threats.  Specifically,  the  RMS  program  has  not  yet 
demonstrated  that  the  system,  operating  in  its  tactical  single 
pass  search  modes,  can  meet  its  detection  and  classification 
requirements  against  deep  water  targets  moored  near  the 
ocean  bottom,  near-surface  moored  mines  that  can  only 

be  detected  by  the  Airborne  Laser  Mine  Detection  System 
in  very  clear  waters,  or  stealthy  bottom  mines.  Unless 
corrected,  these  problems  will  likely  affect  the  quality 
LCS-based  minehunting  and  mine  clearance  operations 
adversely  in  some  threat  scenarios.  As  an  alternative, 
additional  RMS  search  passes  that  will  negatively  affect  the 
efficiency  of  minehunting  and  mine  clearance  operations 
might  be  required  in  some  cases. 

•  The  results  of  developmental  and  integrated  testing  to  date 
continue  to  show  that  the  RMS’s  AN/AQS-20A  sensor  does 
not  meet  Navy  requirements  for  contact  depth  localization 
accuracy  or  false  classification  density  (number  of  contacts 
erroneously  classified  as  mine-like  objects  per  unit  area 
searched).  Contact  depth  localization  problems  complicate 
efforts  to  complete  identification  and  neutralization  of 
mines.  False  classifications,  unless  eliminated  from  the 
contact  list,  require  identification  and  neutralization  effort. 
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result  in  the  expenditure  of  limited  neutralizer  assets,  and 
negatively  affect  the  LCS  sustained  area  coverage  rate.  To 
mitigate  the  problem  of  false  classifications,  the  Navy  has 
implemented  tactics  and  software  designed  to  compare 
the  results  of  multiple  search  passes  over  the  same  area  to 
“prune  out”  most  false  classifications  and  minimize  the 
number  conveyed  for  identification/neutralization.  Under 
some  conditions,  the  Navy  has  demonstrated  that  these 
pruning  tactics  reduce  false  classification  densities  to  the 
Navy's  acceptable  limits.  However,  as  observed  during 
developmental  testing  in  1QFY15,  these  new  procedures 
do  not  reduce  false  classification  densities  appreciably 
in  all  operationally  relevant  conditions.  The  continued 
need  for  additional  passes  to  “prune  out”  excessive 
classifications  will  prevent  the  LCS  MCM  mission  package 
from  achieving  the  Navy’s  predictions  for  Sustained  Area 
Coverage  Rates  that  were  based  on  the  expectation  that 
RMS  would  be  a  “single-pass”  system. 

•  The  Navy  is  developing  AN/AQS-20  pre-planned 
product  improvements  (P3I)  as  a  longer-term  solution  to 
improve  probability  of  correct  classification,  reduce  false 
classifications,  and  resolve  contact  localization  accuracy 
problems.  In  early  FY15,  the  Navy  was  optimistic  that  it 
could  produce  a  mature  P3I  system  prior  to  the  first  phase 
of  LCS  MCM  operational  testing  originally  planned  in 
late  FY15.  The  Program  Office  now  expects  the  improved 
AN/AQS-20C  system  to  enter  operational  testing  in  FY18. 

•  Developmental  testing  of  the  RMS  in  2008  revealed  that 
the  system  had  problems  reacquiring  bottom  objects 

for  identification  in  deeper  waters.  Although  the  Navy 
implemented  fixes  in  the  v6.0  RMMV  designed  to  correct 
this  deficiency,  the  Navy  has  not  yet  conducted  sufficient 
testing  to  evaluate  the  efficacy  of  its  fix. 

•  During  an  AN/AQS-20A  Operational  Assessment  in  2012, 
operators  had  difficulty  identifying  bottom  objects  in  areas 
with  degraded,  but  operationally  relevant,  water  clarity. 
Unless  system  performance  in  this  environment  improves, 
degraded  water  clarity  will  delay  MCM  operations. 

•  Current  tactics  indicate  the  RMS  will  survey  its  tasking 
area  multiple  times  before  sailors  are  able  to  determine  the 
absence  or  presence  of  mines  or  complete  mine  clearance 
operations.  Following  an  initial  search  by  the  RMS,  tactics 
advise  sailors  to  plan  additional  RMS  sorties  to  assess 
persistence  of  in-volume  contacts  marked  as  mine-like  and 
to  identify  bottom  contacts  marked  as  mine-like  as  either 
mines  or  non-mines.  When  operators  conclude  that  RMS 
in-volume  contacts  are  persistent,  those  contacts  are  passed 
to  a  follow-on  system  for  identification/neutralization. 

•  Although  DOT&E’s  analysis  of  RMS  data  collected 
during  LCS  2  and  Increment  1  MCM  mission  package 
TECHEVAL  is  still  in  progress,  preliminary  results  indicate 
that  the  RMS  and  its  operators  made  multiple  mine-like 
calls  on  some  mines.  This  is  an  expected  result  when  the 
sonar  has  multiple  opportunities  to  detect  the  same  mine 


in  favorable  conditions.  The  Navy’s  contact  management 
tool  is  designed  to  post-process  and  compare  the  positions 
of  the  mine-like  calls  generated  by  multiple  opportunities 
to  produce  a  list  of  unique  contact  positions  for  follow-on 
action.  During  TECHEVAL,  however,  the  Navy  noted 
multiple  cases  where  more  than  one  RMS  contact  was 
generated  for  a  single  mine,  passed  through  the  contact 
management  tool,  and  assigned  to  the  Airborne  Mine 
Neutralization  System  (AMNS)  on  multiple  identification 
and  neutralization  attempts.  In  most  of  these  cases,  LCS  2 
conducted  additional,  and  unnecessary,  AMNS  attack 
runs  even  after  the  mine  was  successfully  identified  and 
neutralized.  If  large  numbers  of  duplicate  classifications 
are  passed  to  the  AMNS  for  follow-on  action,  LCS  will 
expend  needless  resources  and  mine  clearance  rates  will  be 
reduced. 

Communications 

•  Two  significant  communications  shortcomings  limit  the 
effectiveness  of  the  current  LCS  MCM  mission  package 
system-of-systems.  One  is  the  limited  range  of  high  data 
rate  communications  between  an  off-board  RMMV  and 
the  host  LCS  and  the  other  is  related  to  the  persistent 
difficulty  with  establishing  and  maintaining  the  existing 
LOS  and  OTH  communications  channels.  The  former 
limits  the  reach  and  productivity  of  LCS  MCM  operations, 
and  the  latter  results  in  frequent  mission  delays  and  the 
potential  loss  of  an  RMMV  with  which  the  LCS  is  unable  to 
communicate.  Unless  these  problems  are  solved,  the  LCS 
and  its  MCM  mission  package  will  never  be  able  to  fulfill 
its  wartime  MCM  missions  within  the  timelines  required. 

•  Although  the  RMMV  can  search  autonomously  while 
operating  OTH  from  the  LCS,  it  can  only  conduct 
electro-optical  identification  operations  to  reacquire 
and  identify  bottom  mines  when  operating  within  LOS 
communications  range  of  the  LCS.  This  limitation  will 
complicate  MCM  operations  in  long  shipping  channels,  and 
will  make  it  necessary  to  clear  a  series  of  LCS  operating 
areas  to  allow  the  ship  to  follow  MCM  operations  as 

they  progress  along  the  channel.  The  cleared  operating 
areas  must  be  close  enough  to  the  intended  search  area  to 
maintain  LOS  communications  and  large  enough  to  enable 
LCS  operations,  including  ship  maneuvering  to  facilitate 
launch  and  recovery  of  the  RMMV  and  MH-60S  helicopter. 
The  additional  time  required  to  clear  these  areas  will 
increase  the  demand  for  mine  clearance.  Although  a  May 
2012  Navy  briefing  proposed  development  of  an  airborne 
relay  and  a  high  frequency  ground  wave  radio  capability, 
along  with  other  upgrades,  to  make  the  Increment  1  MCM 
mission  package  “good  enough”  for  IOT&E,  the  Navy 
has  not  yet  fielded  either  of  those  capabilities.  Had  LCS 
2  been  required  to  clear  its  operating  areas  during  the 
2015  TECHEVAL  and  the  Area  Coverage  Rate  Sustained 
remained  unchanged,  the  time  required  to  complete  MCM 
operations  in  the  test  field  would  have  increased  nearly 
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three-fold.  In  the  May  2012  briefing  cited  above,  the  Navy 
reached  a  similar  conclusion  regarding  the  operational 
consequences  of  limited  RMMV  communications  ranges. 

•  During  TECHEVAL,  LCS  2  had  frequent  problems 
establishing  initial  communications  between  the  ship  and 
an  RMMV  using  existing  OTH  and  LOS  channels  and 
maintaining  those  communications  links  once  established. 
These  problems  frequently  delayed  the  start  of  RMS 
missions  and  periodically  terminated  missions  prematurely. 
On  one  occasion,  loss  of  communications  during  an  attempt 
to  launch  an  RMMV  caused  the  ship  to  return  to  port  with 
the  RMMV  suspended  from  the  Twin-Boom  Extensible 
Crane  because  the  crew  was  unable  to  complete  the  launch 
or  bring  the  vehicle  back  into  the  mission  bay.  On  another 
occasion,  loss  of  LOS  communications  resulted  in  extensive 
damage  to  an  RMMV  that  required  months  of  depot-level 
repair  at  the  contractor’s  facility  when  the  ship  attempted  to 
recover  it  using  OTH  communications.  On  a  third  occasion, 
an  abrupt  loss  of  power  led  to  loss  of  communications  with 
an  RMMV,  making  it  necessary  for  a  test  support  craft  to 
take  the  RMMV  under  tow.  In  addition  to  these  incidents, 
the  LCS  crew  routinely  found  it  necessary  to  seek  help 
from  shore -based  technicians  to  resolve  communications 
problems.  During  the  latter  portion  of  TECHEVAL,  the 
program  manager  embarked  a  team  of  subject  matter 
experts  to  monitor  LCS-RMMV  communications,  assist 
with  troubleshooting,  and  collect  diagnostics.  Shortly 
after  the  TECHEVAL,  the  Program  Office  established  a 
task  force  to  analyze  the  communications  problems  and 
propose  solutions.  The  task  force  has  since  recommended 

a  multi-faceted  approach  that  includes  improving  operating 
and  troubleshooting  documentation  for  the  communications 
systems,  enhanced  crew  training  for  initializing  of 
communications  links  and  fault  troubleshooting,  and,  longer 
term,  a  reexamination  of  the  communications  architecture. 

Launch  and  Recovery 

•  The  Independence  variant  LCS  has  had  difficulty  launching 
and  recovering  the  RMMV  because  of  the  vehicle’s  erratic 
motion  in  the  ship’s  wake.  In  past  developmental  testing, 
violent  RMMV  yaw  and  roll  motions  have  overstressed  and 
damaged  the  launch  and  recovery  hardware  and  resulted  in 
damage  to  the  RMMV.  Although  the  Navy  implemented 
materiel,  training,  and  procedural  improvements,  incidents 
of  equipment  damage  and  launch  and  recovery  failures 
continue  to  delay  or  prevent  sustained  operations.  The 
Navy  completed  16  RMMV  launches  and  14  RMMV 
recoveries  during  23  days  at  sea  in  developmental  testing 
conducted  in  1QFY15.  During  58  days  at  sea  during 
TECHEVAL,  both  the  pace  and  success  rate  of  RMMV 
launch  and  recovery  regressed  as  LCS  2  completed  24 
RMMV  launches  and  only  18  RMMV  recoveries.  The 
increased  frequency  of  unrecovered  RMMVs  is  attributed  to 
a  larger  number  of  off-board  vehicle  failures  that  precluded 
recovery  aboard  LCS  2  rather  than  new  launch  and  recovery 
problems.  Damage  to  shipboard  launch  and  recovery 
equipment,  LCS-RMMV  communications  problems, 


multiple  RMMV  hydraulics  system  failures,  a  suspected 
RMMV  electrical  system  failure,  and  RMMV  mast  latch 
and  fuel  system  failures  contributed  to  the  ship’s  inability 
launch  or  recover  the  unmanned  vehicle. 

•  No  RMMV  launch  and  recovery  operations  have  been 
conducted  aboard  a  Freedom  variant  LCS  at  sea. 

RMS  Vulnerabilities 

•  The  combination  of  acoustic  radiated  noise,  frequent 
RMMV  failures  that  prevent  recovery  aboard  LCS,  and  the 
probability  the  vehicle  and  its  sensor  will  get  entangled  with 
mines  or  other  hazards  all  pose  a  risk  to  losing  the  RMS. 
Given  the  limited  existing  inventory  of  RMMVs  (four  v6.0 
vehicles,  four  vehicles  awaiting  upgrades  to  v6.0,  and  two 
vehicles  designated  for  training  use  only),  any  additional 
RMMV  attrition  would  severely  degrade  the  Navy’s  ability 
to  conduct  LCS-based  MCM  operations. 

•  RMMV  acoustic  radiated  noise  measurements,  last 
collected  during  developmental  testing  in  2007/2008, 
indicated  that  existing  RMMVs  might  be  vulnerable  to 
some  mines.  The  RMS  Program  Office  has  not  assessed 
radiated  noise  following  recent  vehicle  configuration 
changes  and  has  requested  a  waiver  to  deploy  the  system 
even  through  it  did  not  previously  meet  its  acoustic  radiated 
noise  specification.  If  RMMV  radiated  noise  continues 

to  exceed  acceptable  limits,  systems  could  be  lost  during 
LCS-based  minehunting  and  mine  clearance  operations 
depleting  the  Navy's  limited  inventory  of  assets.  The 
magnetic  signature  of  the  v6.0  RMMV  has  not  been 
measured. 

•  As  noted  earlier,  only  18  of  24  RMMVs  launches  from  LCS 
2  ended  with  an  RMMV  recovery  aboard  LCS  2  during 
TECHEVAL.  Frequent  RMMV  failures  that  preclude 
vehicle  recovery  aboard  LCS  might  result  in  lost  RMMVs 
and  expose  personnel  who  attempt  to  recover  RMMVs  in 
open  waters  to  air,  surface,  and  mine  threats.  Because  of 
the  number  of  incidents  in  which  an  RMMV  could  not  be 
recovered,  the  Navy  is  now  considering  options  that  would 
provide  LCS  with  additional  support  to  recover  RMMVs 
that  it  cannot  recover  otherwise.  On  four  occasions  during 
TECHEVAL,  RMMV  failures  precluded  LCS  2  from 
controlling  the  movements  of  an  off-board  RMMV.  If 
similar  failures  occur  during  operations,  the  RMMV  could 
become  disabled  in  the  minefield  or  drift  into  a  minefield 
before  salvage  or  support  craft  arrive  to  recover  it. 

•  Even  though  test  minefields  are  deliberately  planned  to 
reduce  the  risk  of  RMS  striking  a  mine  target  or  becoming 
entangled  in  its  mooring  cable,  the  RMS  has  snagged 
several  tethered  mines,  and  other  surface  and  underwater 
objects  during  testing.  These  incidents  often  cause  damage 
to  the  vehicle  or  deployed  sonar  that  leaves  the  system 
inoperable.  In  some  cases,  divers  embarked  on  test  support 
craft  have  entered  the  water  to  assist  in  recovery  of  assets 
following  a  snag.  Although  the  Navy  is  still  developing 
concept  of  operations  to  handle  these  situations  during 
operations  in  a  threat  minefield,  it  is  clear  that  if  these 
incidents  occur  during  wartime  operations  they  will 
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pose  a  risk  to  vehicles  and  potential  recovery  personnel. 
Furthermore,  the  repeated  occurrence  of  these  incidents 
presents  both  a  tactical  and  a  system  design  challenge  for 
the  Navy  as  it  tries  to  minimize  attrition  when  the  system  is 
employed  operationally. 

•  In  FY15,  the  Navy  also  disclosed  that  the  AN/AQS-20A 
does  not  trail  directly  behind  the  RMMV  when  deployed 
to  tactical  minehunting  depths.  Instead,  the  sensor  tows  to 
starboard  of  the  RMMV  path.  This  offset  causes  the  RMS 
to  behave  like  a  mine  sweeping  system  as  the  sonar  and  its 
tow  cable  passes  through  the  water,  thereby  increasing  the 
risk  of  snagging  a  tethered  mine. 

•  The  RMS  Program  has  not  completed  the  final  Adversarial 
Assessment  phase  of  cybersecurity  operational  testing  of 
the  RMS  hardware  and  software  configurations  intended 
for  Initial  Operational  Capability  in  the  LCS  MCM  mission 
package  in  FY16. 

Maintainability 

•  Consistent  with  the  concept  of  operations,  the  LCS  is  reliant 
on  shore -based  support  for  assistance  with  diagnosis  and 
repair  of  seaframe  equipment  and  MCM  system  problems. 
Although  the  ship  could  be  more  self-reliant  if  the  sailors 
were  provided  with  better  maintenance  training,  technical 
documentation,  test  equipment,  and  tools  and  a  more 
extensive  stock  of  spares,  the  mission  package  detachment 
lacks  the  wherewithal  to  handle  anything  beyond  relatively 
uncomplicated  RMS  preventive  maintenance  and  minor 
repairs.  As  a  result,  the  Navy’s  records  show  that  shore- 
based  RMMV  maintenance  personnel  completed  more  than 
4,000  hours  of  RMMV  maintenance  over  six  months  of 
TECHEVAL  work-ups  and  testing  to  support  approximately 
108  hours  of  RMS  minehunting.  Not  only  is  this  level  of 
support,  38  hours  of  maintenance  per  hour  of  minehunting, 
far  beyond  the  capability  of  the  embarked  crew,  it  is  also 
not  sustainable  for  wide-area  LCS  MCM  operations  that 
must  be  quickly  completed. 

Recommendations 

•  Status  of  Previous  Recommendations. 

-  The  Navy  made  progress  on  all  four  FY13 
recommendations.  Shore-based  testing  completed  in 
1QFY14  and  shipboard  testing  completed  in  1QFY15 
provided  additional  information  regarding  RMS,  RMMV, 
and  AN/AQS-20A  reliability;  RMS  operational  availability; 
and  RMMV  launch,  handling,  and  recovery  system 
performance.  Although  the  Navy  continues  to  develop  and 
test  AN/AQS-20A  upgrades,  it  has  not  demonstrated  in 
developmental  or  operational  testing  that  it  has  corrected 
problems  with  false  classifications  and  contact  localization 
errors  that  will  otherwise  limit  performance  in  operational 
testing.  The  Navy  expects  to  complete  its  update  to  the 
RMS  TEMP,  which  now  includes  the  AN/AQS-20A  sonar, 
in  FY16. 

-  The  Navy  has  made  progress  on  two  of  the  nine  FY14 
recommendations.  The  Navy  initiated  RMS  cybersecurity 


and  conducted  additional  ship-based  RMS  testing  to 
assess  readiness  for  operational  testing  that  it  expected  to 
complete  in  FY15.  The  Navy  did  not  address  the  following 
FY14  recommendations: 

1 .  Identify  the  RMS  configuration  for  operational  testing  of 
LCS  equipped  with  the  first  increment  of  MCM  capability 
and  complete  the  required  operationally  realistic  testing 
of  that  system  prior  to  LCS  MCM  mission  package 
TECHEVAL. 

2.  When  system  maturity  is  able  to  support,  conduct  testing  of 
the  RMS  consisting  of  the  v6.0  RMMV  and  AN/AQS-20C 
in  operationally  realistic  end-to-end  minehunting  missions 
to  characterize  AN/AQS-20B  minehunting  performance  and 
accurately  assess  availability  of  the  RMS  and  reliability  of 
the  RMMV  and  AN/AQS-20B. 

3.  Investigate  the  use  of  communications  relays  and  other 
solutions  that  might  improve  the  standoff  distance  between 
an  RMMV  and  its  host  ship  to  improve  the  efficiency  of 
LCS  MCM  operations. 

4.  Document  a  robust  reliability  monitoring  and  growth 
strategy  for  any  new  low-rate  initial  production  vehicles 
procured  following  a  planned  FY15  Milestone  C  decision. 

5.  Reassess  v6.0  RMMV  radiated  noise  following  vehicle 
upgrades. 

6.  Reexamine  minimum  vehicle  and  sensor  reliability  and 
LCS  organizational-level  maintenance  support  needed  to 
complete  timely  and  realistic  operational  scenarios  without 
excessive  reliance  on  intermediate-  and  depot-level  support. 

7.  Reconsider  RMS  minehunting  requirements  in  the  context 
of  expected  LCS  tactics  and  operations. 

•  FY 15  Recommendations.  In  addition  to  addressing 

outstanding  FY14  recommendations,  the  Navy  should: 

1.  Review  RMMV  design  alternatives  as  a  solution  for  system 
reliability  problems. 

2.  Complete  a  comprehensive  review  of  RMMV  and  mission 
package  communications  interfaces  and,  if  necessary, 
re-engineer  the  Multi- Vehicle  Communication  System, 
RMMV,  and/or  other  essential  system-of-systems 
components  to  improve  interoperability  and  enable  reliable 
LOS  and  OTH  communications  between  LCS  and  RMMVs. 

3.  Develop  tactics  to  mitigate  system  vulnerabilities  to  mines, 
mine  collision,  and  entanglement  hazards,  and  other  surface 
and  underwater  hazards. 

4.  Assess  improvements  to  post  mission  analysis  and  contact 
management  software  and  training  to  resolve  problems 
observed  during  TECHEVAL  when  multiple  RMS  contacts 
on  the  same  mine  were  passed  to  AMNS  for  identification 
and  neutralization. 

5.  Continue  to  develop  and  implement  improvements  for 
launch,  handling,  RMMV  and  recovery  equipment  and 
procedures. 

6.  Provide  LCS  sailors  better  training,  technical 
documentation,  test  equipment,  and  tools,  along  with 
additional  spares  to  improve  the  crews’  self-sufficiency  and 
enhance  RMS  maintainability. 
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Rolling  Airframe  Missile  (RAM)  Block  2 


Executive  Summary 

•  Eight  Rolling  Airframe  Missile  (RAM)  Block  2  developmental 
missile  firings  have  been  conducted  in  addition  to 

17  IOT&E  missile  firings  the  Commander,  Operational 
Test  and  Evaluation  (COTF)  conducted  from  March  2013 
through  April  2015,  at  the  Naval  Air  Warfare  Center, 

Point  Mugu,  California,  and  the  Naval  Weapons  Station, 
Yorktown,  Virginia.  IOT&E  testing  was  conducted  in 
accordance  with  a  DOT&E-approved  test  plan. 

•  The  results  of  developmental  and  operational  testing 
completed  to  date  indicate: 

-  RAM  Block  2  effectiveness  is  comparable  to  RAM 
Block  lA’s  effectiveness  against  older  anti-ship  cruise 
missiles  (ASCM)  threats.  However,  DOT&E  cannot 
make  a  final  determination  of  RAM  Block  2  effectiveness 
against  newer  ASCM  threats  until  completion  of  IOT&E  in 
mid  2017. 

-  RAM  Block  2  has  an  improved  kinematic  capability  to 
guide  on  maneuvering  ASCMs  and  an  improved  capability 
to  guide  on  certain  ASCM  threat  emitters  over  RAM 
Block  1A. 

-  RAM  Block  2  demonstrated  satisfactory  missile  reliability 
with  no  confirmed  reliability  failures  in  the  25  RAM 
Block  2  firings  from  the  Self-Defense  Test  Ship  (SDTS). 

-  Deficiencies  in  RAM  Block  2  integration  with  the  Ship 
Self-Defense  System  (SSDS)-based  combat  system  caused 
several  RAM  Block  2  missiles  to  miss  their  target  during 
one  of  the  IOT&E  missile  firing  scenarios. 

-  No  assessment  of  RAM  Block  2’s  capability  against 
Multi-Stage  Supersonic  Target  (MSST)-like  ASCM  threats 
is  possible  due  to  the  lack  of  an  MSST. 

-  RAM  Block  2  demonstrated  lethality  comparable  to 
Block  1 A  and  Block  0. 

•  DOT&E  is  currently  preparing  an  Early  Fielding  Report  to 
Congress  on  the  completed  testing  and  will  conduct  a  full 
assessment  of  RAM  Block  2  effectiveness,  suitability,  and 
lethality  after  IOT&E  is  completed. 

System 

•  The  RAM,  jointly  developed  by  the  United  States  and 
the  Federal  Republic  of  Germany,  provides  a  short-range, 


lightweight,  self-defense  system  to  defeat  ASCMs.  There  are 
three  RAM  variants: 

-  RAM  Block  0  uses  dual  mode,  passive  radio 
frequency/ infrared  guidance  to  home  in  on  ASCMs. 

-  RAM  Block  1 A  adds  infrared  guidance  improvements  to 
extend  defenses  against  ASCMs  that  do  not  radiate  radio 
frequencies. 

-  RAM  Block  2  incorporates  changes  to  improve  its 
kinematic  capability  and  capability  to  guide  on  certain 
types  of  ASCM  radio  frequency  threat  emitters  in  order  to 
defeat  newer  classes  of  ASCM  threats. 

Mission 

•  Naval  component  commanders  will  use  RAM  to  accomplish 
ship  self-defense  missions. 

•  Naval  surface  forces  will  use  RAM  to  provide  a  short-range, 
hard-kill  engagement  capability  against  ASCM  threats. 

Major  Contractors 

•  Raytheon  Missiles  Systems  -  Tucson,  Arizona 

•  RAMSys  -  Ottobrunn,  Germany 


Activity 

•  The  Navy  conducted  8  RAM  Block  2  developmental  missile 
firings  and  17  IOT&E  missile  firings  from  the  SDTS  at  the 
Naval  Air  Warfare  Center,  Point  Mugu,  California,  from 
March  2013  through  April  2015.  COTF  conducted  the  IOT&E 
missile  firings  in  accordance  with  a  DOT&E-approved  test 
plan. 


COTF  conducted  an  ammunition  on-load  demonstration 
as  part  of  IOT&E  onboard  USS  Arlington  (LPD  24)  in 
April  2015  at  the  Naval  Weapons  Station,  Yorktown,  Virginia. 
RAM  Block  2  IOT&E  for  at-sea  testing  is  ongoing  and  is 
scheduled  for  completion  in  mid-2016. 
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•  COTF  continued  planning  for  the  Probability  of  Raid 
Annihilation  Modeling  and  Simulation  test  bed  IOT&E  test 
phase.  This  testing  is  scheduled  to  commence  in  late-2016. 

•  DOT&E  is  currently  preparing  an  Early  Fielding  Report  to 
Congress  on  the  completed  testing  and  will  conduct  a  full 
assessment  of  RAM  Block  2  effectiveness,  suitability,  and 
lethality  after  IOT&E  is  completed. 

Assessment 

•  Results  of  testing  completed  to  date  indicate  that 

RAM  Block  2  performance  is  comparable  to  the  RAM  Block 
1 A  performance  against  older  ASCM  threats.  However, 
DOT&E  cannot  make  a  final  determination  of  RAM  Block 
2  performance  against  newer  ASCM  threats  until  IOT&E  is 
completed  in  mid-2017. 

•  RAM  Block  2  demonstrated  an  improved  kinematic  capability 
and  an  improved  capability  to  guide  on  certain  ASCM  threat 
emitters  over  RAM  Block  1  A. 

•  RAM  Block  2  demonstrated  satisfactory  missile  reliability 
with  no  confirmed  reliability  failures  in  the  25  RAM  Block  2 
firings  from  the  SDTS  to  date. 

•  RAM  Block  2  demonstrated  lethality  comparable  to  Block  1A 
and  Block  0. 

•  Deficiencies  in  RAM  Block  2  integration  with  the  SSDS-based 
combat  system  caused  several  RAM  Block  2  missiles  to  miss 
their  target  during  one  of  the  IOT&E  missile  firing  scenarios. 
The  Navy  has  initiated  a  formal  Failure  Review  Board  to 
determine  the  required  corrections. 

•  The  CVN  and  LHA  6  class  ships  defend  themselves  against 
ASCMs  by  first  using  the  medium-range  Evolved  SeaSparrow 
Missile  (ESSM)  and  then  the  shorter-range  RAM.  RAM  uses 
radio  frequency  and/or  infrared  terminal  guidance  to  home  on 
ASCM  threats.  Hot  debris  from  prior  intercepts  and  warhead 
detonations  can  therefore  interfere  with  RAM’s  infrared 
guidance.  While  the  SSDS  is  designed  to  schedule  RAM  and 
ESSM  engagements  to  avoid  this  type  of  interference,  it  failed 
to  do  so  during  testing. 


•  The  AN/SLQ-32  Electronic  Warfare  System  (EWS)  with  the 
Surface  Electronic  Warfare  Improvement  Program  (SEWIP) 
Block  1  upgrade  was  not  able  to  timely  detect  certain  types  of 
ASCM  emitter  signals.  The  late  detections  negatively  affected 
the  performance  of  RAM  missiles  that  the  SSDS  employed 
against  these  ASCM  threats.  The  Navy  is  addressing  this 
deficiency  with  the  SEWIP  Block  2  upgrade  to  the  AN/SLQ-32 
EWS. 

•  Two  BQM-74  aerial  targets  failed  to  maintain  operationally 
realistic  flight  parameters  in  one  of  the  IOT&E  missile  firing 
scenarios. 

•  Due  to  the  lack  of  an  MSST,  no  assessment  of  RAM  Block  2’s 
capability  against  MSST-like  ASCM  threats  is  possible. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Complete  all  planned  RAM  Block  2  missile  firing  IOT&E 
test  events. 

2.  Correct  the  identified  integration  deficiencies  with 
the  SSDS-based  combat  system  and  RAM  Block  2. 
Demonstrate  these  corrections  in  a  phase  of  operational 
testing. 

3.  Correct  the  SSDS  scheduling  function  to  preclude 
interference  from  prior  intercepts  and  warhead  detonations 
with  RAM’s  infrared  guidance.  Demonstrate  corrections  in 
a  phase  of  operational  testing. 

4.  Investigate  why  the  BQM-74  aerial  targets  failed  to 
maintain  operationally  realistic  flight  parameters. 
Demonstrate  any  corrections  prior  to  using  these  targets  in 
similar  operational  test  scenarios. 

5.  Continue  planning  for  the  Probability  of  Raid  Annihilation 
Modeling  and  Simulation  IOT&E  test  phase. 
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RQ-21 A  Blackjack  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force,  with  assistance  from  the  Marine  Corps  Operational 
Test  and  Evaluation  Activity,  conducted  the  RQ-21AIOT&E 
between  January  and  December  2014  in  three  phases  at 
Twentynine  Palms,  California;  Camp  Lejeune,  North  Carolina; 
and  aboard  USS  Anchorage  (LPD  23).  Total  flight  time 
accrued  from  all  three  test  locations  was  233.6  hours  during 
38  flights. 

•  DOT&E  submitted  an  IOT&E  report  in  June  2015  based 
on  the  demonstrated  performance  during  IOT&E  to  inform 
the  Services’  Full-Rate  Production  (FRP)  decision,  which 
was  originally  planned  for  October  2015,  now  scheduled 
for  September  2016.  The  Navy  plans  to  conduct  additional 
developmental  testing  and  is  investigating  the  conduct 

of  a  Verification  of  Correction  of  Deficiencies  before  the 
September  2016  FRP  decision.  DOT&E  intends  to  provide 
an  updated  assessment  prior  to  the  FRP  decision  to  further 
support  that  decision.  In  the  IOT&E  report,  DOT&E 
concludes: 

-  The  RQ-21  A  testing  was  adequate  and  conducted  in 
accordance  with  the  DOT&E -approved  test  plan. 

-  The  RQ-21  A  is  not  operationally  effective. 

-  The  RQ-21  A  is  not  operationally  suitable. 

-  The  system  has  exploitable  cybersecurity  vulnerabilities. 

System 

•  Each  system  consists  of  five  RQ-21  A  unmanned  aircraft, 
surface  components,  and  assorted  government-provided 
equipment: 

-  The  surface  components  consist  of  two  ground  control 
stations,  launch  and  recovery  equipment,  datalinks, 
multi-mission  payloads,  and  support  systems. 

-  Government  provided  equipment  includes  vehicles  and 
generators  to  transport  and  power  ground  components  as 
well  as  intelligence  workstations. 

•  The  Marine  Corps  intends  the  RQ-21  A  system  to  have: 

-  The  reliability  to  support  an  operating  tempo  of  12  hours 
on  station  per  day  at  a  sustained  rate  for  30  days  and  the 
capability  for  one  surge  of  24  hours  on-station  coverage 
per  day  for  a  10-day  period  during  any  30-day  cycle 

-  An  aircraft  with  10  hours  endurance,  airspeed  up  to 

80  nautical  miles  per  hour,  a  service  ceiling  of  15,000  feet 


density  altitude,  and  an  operating  radius  of  50  nautical 
miles 

-  An  electro-optical  sensor  capable  of  providing  the  ground 
control  station  operator  team  sufficient  visual  resolution 
to  support  classification  of  a  1 -meter  linear  sized  object 
from  3,000  feet  altitude  above  ground  level  and  a  sensor 
depression  angle  of  45  degrees,  resulting  in  an  assessment 
at  a  slant  range  of  4,242  feet 

-  An  infrared  sensor  capable  of  classifying  a  3-meter  sized 
linear  object  from  3,000  feet  above  ground  level  and  slant 
range  of  4,242  feet 

-  An  entire  system  transportable  by  CH-53E  helicopter 

Mission 

•  Marine  Corps  commanders  will  use  the  RQ-21  A  Blackjack  to 
provide  units  ashore  with  a  dedicated  battlefield  Intelligence, 
Surveillance,  and  Reconnaissance  (ISR)  capability  that  will 
reduce  their  dependence  on  higher  headquarters  for  ISR 
support. 

•  Detachments  from  Marine  Corps  Unmanned  Aircraft  System 
Squadrons  will  embark  the  requisite  personnel  and  equipment 
aboard  L-class  ships  and  will  be  capable  of  conducting  ship  to 
shore  operations. 

Major  Contractor 

Insitu,  Inc.  -  Bingen,  Washington 


Activity 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force,  with  assistance  from  the  Marine  Corps  Operational 
Test  and  Evaluation  Activity,  conducted  the  RQ-21  A  IOT&E 


between  January  and  December  2014  in  three  phases  at 
Twentynine  Palms,  California;  Camp  Lejeune,  North 
Carolina;  and  aboard  USS  Anchorage  (LPD  23).  Total 
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flight  time  accrued  from  all  three  test  sites  was  233.6  hours 
during  38  flights.  The  Navy  and  Marine  Corps  conducted  the 
RQ-21 A  testing  in  accordance  with  the  DOT&E-approved  test 
plan. 

•  In  May  2014,  the  Marine  Corps  deployed  an  Early  Operational 
Capability  RQ-21  A  system  to  support  overseas  combat 
operations.  This  deployed  detachment  consisted  of  up  to 
eight  aircraft,  two  launchers,  and  two  recovery  systems. 
Between  May  and  September,  the  system  flew  approximately 
121  flights  for  995  flight  hours.  Data  collected  during  the 
Early  Operational  Capability  detachment’s  deployment 
overseas  provided  additional  insights  into  system  reliability. 

•  In  June  2015,  DOT&E  submitted  an  IOT&E  report  based 
on  the  demonstrated  performance  during  testing  to  inform 
the  Services’  FRP  decision,  which  was  originally  planned 
for  October  2015,  now  scheduled  for  September  2016.  The 
Navy  plans  to  conduct  additional  developmental  testing  and 
is  investigating  the  conduct  of  a  Verification  of  Correction 
of  Deficiencies  before  the  September  2016  FRP  decision. 
DOT&E  intends  to  provide  an  updated  assessment  prior  to  the 
FRP  decision  to  further  support  that  decision. 

Assessment 

•  The  detachment  equipped  with  RQ-21  A  is  not  effective  in 
supporting  the  ground  commander’s  mission  because  of  an 
inability  to  have  an  unmanned  aircraft  arrive  on  station  at  the 
designated  time  and  remain  on  station  for  the  duration  of  the 
tasked  period.  During  the  IOT&E,  the  RQ-21  A-equipped  unit 
provided  coverage  during  68  percent  of  the  tasked  on-station 
hours  (83.8  of  122.7  hours). 

•  The  c I  cc ti'o -o pt i  ca I  i n  frare d  sensor  provides  accurate  target 
locations.  While  the  Capabilities  Production  Document 
does  not  specify  a  threshold  value  for  sensor  point  of  interest 
accuracy,  Marine  Corps  guidance  indicates  that  100  meter 
accuracy  is  sufficient  to  support  tactical  operations.  RQ-21  A 
provides  a  90-percent  circular  error  probable  target  location 
error  of  43.8  meters.  Such  accuracy  is  sufficient  to  support 
targeting  in  a  conventional  linear  battlefield,  but  does  not 
support  targeting  in  a  dense  urban  environment  that  requires 
more  accurate  target  locations. 

•  The  RQ-21  A  sensor  does  not  meet  one  of  the  two  target 
classification  Key  Performance  Parameters  (KPPs)  established 
in  the  Capabilities  Production  Document.  The  electro-optical 
sensor  does  not  provide  a  50  percent  probability  of  correct 
classification  for  1-meter  linear  objects  (weapons  or  tools). 

The  infrared  sensor  does  meet  the  50  percent  threshold 
probability  for  correctly  classifying  3-meter  objects 
(vehicle  chassis  type)  by  demonstrating  100  percent  correct 
classification. 

•  The  communications  relay  payload  limits  the  commanders’ 
tactical  flexibility  and  mission  accomplishment.  It  is 
constrained  to  a  single  frequency  in  each  of  the  two  radios  that 
are  set  before  launch.  Once  airborne,  operators  cannot  change 
frequencies.  This  limits  the  communications  relay  payload 

to  supporting  pre-planned  relay  missions  on  a  single  pair  of 
pre-set  frequencies. 
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•  Aircraft  endurance  meets  the  10-hour  KPP.  The  longest 
IOT&E  flight  lasted  12.3  hours. 

•  The  recessed,  nose-mounted  electro-optical/infrared  payload 
requires  circular  orbits  over  the  top  of  the  target  to  maintain 
continuous  coverage  and  positive  target  identification.  The 
use  of  offset  orbits  results  in  the  fuselage  blocking  the  payload 
field  of  view  for  significant  periods  of  time.  These  offset 
orbits  resulted  in  auto-track  break  locks  and  loss  of  positive 
identification  of  high-value  targets.  There  are  orbit  shapes 
that  would  allow  RQ-21  A  operators  to  maintain  continuous 
coverage  of  a  target,  but  the  current  RQ-21  A  operating  system 
limits  operators  to  circular  orbits. 

•  The  RQ-21  A  is  not  operationally  suitable.  TheRQ-21A 
demonstrated  a  Mean  Flight  Hour  Between  Abort  for  the 
System  of  15.2  hours  versus  the  50-hour  requirement. 

Because  of  aircraft  reliability,  overall  system  availability  did 
not  meet  the  80  percent  KPP  threshold  (demonstrated  value 
equals  66.9  percent). 

•  The  average  time  between  overhaul  of  the  propulsion  modules 
was  48.9  hours,  which  does  not  meet  the  manufacturer’s  stated 
100-hour  capability. 

•  The  RQ-21  A  Naval  Air  Training  and  Operations 
Standardization  manual  is  missing  important  information 
regarding  mission  computer  logic.  This  lack  of  information 
is  especially  critical  during  emergencies  where  operators  are 
unaware  of  which  conditions  enable/disable  various  aspects 
of  aircraft  functionality.  This  lack  of  system  operations 
information  contributed  to  the  loss  of  an  aircraft  during  the 
first  IOT&E  flight. 

•  To  support  shipboard  operations,  the  Navy  permanently 
installed  some  RQ-21  A  ground  components  (antennae 
interface  modules,  datalink  antennae)  on  selected  ships. 

The  ships’  personnel,  and  not  the  RQ-21  A  detachment,  own 
and  maintain  these  components.  During  IOT&E,  none  of 
the  ship’s  personnel  received  training  on  maintaining  these 
installed  components.  The  ship  did  not  receive  spare  parts, 
maintenance  manuals,  or  wiring  diagrams  with  which  to 
facilitate  repairs. 

•  Field  service  representatives  are  required  to  conduct 
maintenance  in  accordance  with  the  Naval  Aviation 
Maintenance  Program.  Instead  of  using  the  established  Naval 
Aviation  Logistics  Command  Management  Information 
System  to  track  maintenance  and  flight  information,  the  field 
service  representatives  used  a  program  called  Sapphire,  which 
resulted  in  the  accurate  archiving  of  15  of  136  unscheduled 
maintenance  actions.  Because  of  these  inaccuracies, 

the  Marine  Corps  will  not  be  able  to  determine  the  true 
maintenance  burden  associated  with  RQ-21  A  operations. 

•  Extended  logistics  delay  times  and  production  quality 
control  issues  contributed  to  the  system’s  poor  reliability 
and  availability.  In  six  instances,  aircraft  spent  time  in 

a  non-mission  capable  status  while  awaiting  spare  parts. 
Incorrectly  assembled/configured  components  received  from 
the  manufacturer  increase  the  maintenance  time  to  repair  or 
replace  components,  resulting  in  reduced  mission  availability. 

•  The  system  has  exploitable  cybersecurity  vulnerabilities. 
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Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
progress  in  the  FY13  recommendation  to  accelerate  annual 
operating  hours  in  order  to  reach  the  projected  3,300  flight 
hours  sooner  than  2017,  which  would  allow  the  Navy  to 
identify  and  correct  failure  modes  before  committing  to 
buy  a  significant  number  of  systems.  The  remaining  FY13 
recommendation  to  conduct  a  comprehensive  review  of 
RQ-21 A  reliability  versus  requirements  remains  open. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Improve  the  electro-optical  sensor  resolution  to  increase 
probability  of  correct  classification  for  1 -meter  sized 
targets. 

2.  Increase  the  number  of  programmed  aircraft  loiter 
patterns  to  increase  tactical  flexibility  and  reduce  fuselage 
obstruction  of  the  payload. 

3.  Fully  integrate  the  communications  relay  payload  into  the 
aircraft  architecture  to  allow  for  in-flight  frequency  changes 
and  altering  of  other  radio  settings  to  increase  tactical 
utility. 

4.  Implement  the  cybersecurity  recommendations  listed  in  the 
classified  annex  to  the  IOT&E  report  to  improve  system 
security. 

5.  Expand  the  systems  description  and  flight  characteristics 
section  of  the  RQ-21  A  Naval  Air  Training  and  Operations 


Standardization  manual  to  allow  operators  to  safely  react  to 
system  emergencies. 

6.  Increase  propulsion  module  service  life  and  reliability  to 
reduce  maintainer  workload,  the  number  of  spares  required, 
and  operating  costs. 

7.  Increase  production  quality  control  and  implement  thorough 
acceptance  procedures  for  delivered  systems  and  spare 
parts  to  reduce  the  number  of  faulty  items  received  by  fleet 
operators. 

8.  Fully  train  and  provide  ship  personnel  with  technical 
manuals,  wiring  diagrams,  and  spare  parts  related  to 
RQ-21  A  shipboard  components  to  increase  RQ-21  A  full 
mission  capability. 

9.  Require  field  service  representatives  to  utilize  the  Naval 
Aviation  Logistics  Command/Management  Information 
System  to  track  system  maintenance  to  provide  better 
maintenance  burden  data. 

10.  Increase  expenditures  on  spare  parts  to  reduce 
administrative  logistics  delay  times. 

11.  Increase  the  number  of  spare  parts  in  the  pack-up  kit  to 
increase  system  availability. 
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Ship  Self-Defense  for  LHA(R) 


Executive  Summary 

•  The  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  conducted  at-sea  testing  for  the  LHA(R) 
IOT&E  on  the  USS  America  (LHA  6)  in  May  2015  and  the 
Ship  Self-Defense  Test  Ship  (SDTS)  from  December  2014 
through  March  2015.  At-sea  testing  is  scheduled  to  complete 
in  mid-2016.  Testing  was  conducted  in  accordance  with  the 
DOT&E-approved  test  plans. 

•  Results  of  testing  completed  to  date  indicate  that  the 
LFIA(R)  has  some  ship  self-defense  capability  against  older 
anti-ship  cruise  missile  (ASCM)  threats.  The  LHA(R)  ship 
self-defense  performance  against  newer  ASCM  threats  will 
remain  undetermined  until  the  LFIA(R)  Probability  of  Raid 
Annihilation  (P  )  modeling  and  simulation  (M&S)  test  bed 
runs  for  IOT&E  are  completed  in  mid-2017. 

System 

•  Surface  ship  self-defense  for  the  LHA(R)  is  addressed  by 
several  legacy  combat  system  elements  and  five  acquisition 
programs: 

-  Ship  Self-Defense  System  (SSDS) 

-  Rolling  Airframe  Missile  (RAM) 

-  Evolved  Seasparrow  Missile  (ESSM) 

-  Cooperative  Engagement  Capability  (CEC) 

-  Surface  Electronic  Warfare  Improvement  Program 
(SEWIP) 

SSDS 

•  SSDS  is  a  local  area  network  that  uses  open  computer 
architecture  and  standard  Navy  displays  to  integrate  a  surface 
ship’s  sensors  and  weapons  systems  to  provide  an  automated 
detect-track-engage  sequence  for  ship  self-defense. 

•  SSDS  MK  1  is  the  command  and  control  system  for 
LSD-41/49  class  ships. 

•  SSDS  MK  2  has  six  variants: 

-  Mod  1,  used  in  CVN  68  class  aircraft  carriers 

-  Mod  2,  used  in  LPD  17  class  amphibious  ships 

-  Mod  3,  used  in  LHD  7/8  class  amphibious  ships 

-  Mod  4,  used  in  LHA(R)  class  amphibious  ships 

-  Mod  5,  used  in  LSD  41/49  class  amphibious  ships 

-  Mod  6,  in  development  for  CVN  78  class  aircraft  carriers 
RAM 

•  The  RAM,  jointly  developed  by  the  United  States  and  the 
Federal  Republic  of  Germany,  provides  a  short-range,  light 
weight,  self-defense  system  to  defeat  ASCMs. 

•  There  are  three  RAM  variants: 

-  RAM  Block  0  uses  dual-mode,  passive  radio 
frequency/ infrared  guidance  to  home  in  on  ASCMs. 

-  RAM  Block  1 A  adds  infrared  guidance  improvements 
to  extend  defense  against  non-radio-frequency-radiating 
ASCMs. 


Link  Integration 


Combat  Direction 

CEC 


I 


Command  &  Display 


System  ID  Air  Control 


CEC  -  Cooperative  Engagement  Capability 

ESSM  -  Evolved  Sea  Sparrow  Missile 

ID  -  Identification 

RAM  -  Rolling  Airframe  Missile 

SSDS  -  Ship  Self-Defense  System 


-  RAM  Block  2  is  intended  to  extend  the  capability  of  RAM 
Block  1A  against  newer  classes  of  ASCM  threats. 

ESSM 

•  The  ESSM,  cooperatively  developed  among  13  nations,  is  a 
medium-range,  ship-launched,  self-defense  guided  missile 
intended  to  defeat  ASCM,  surface,  and  low-velocity  air 
threats. 

•  The  ESSM  is  currently  installed  on  LHA(R)  and  LHD  8 
amphibious  ships,  DDG  51  Flight  11A  destroyers,  and  CVN  68 
class  aircraft  carriers  equipped  with  the  SSDS  MK  2  Mod  1 
Combat  System. 

•  There  are  two  variants  of  ESSM: 

-  ESSM  Block  1  is  a  semi-active  radar-guided  missile  that  is 
currently  in  service. 

-  ESSM  Block  2  is  in  development  and  will  have 
semi-active  radar  guidance  and  active  radar  guidance. 

CEC 

•  CEC  is  a  sensor  network  with  an  integrated  fire  control 
capability  that  is  intended  to  significantly  improve  battle  force 
air  and  missile  defense  capabilities  by  combining  data  from 
multiple  battle  force  air  search  sensors  on  CEC-equipped  units 
into  a  single,  real-time,  composite  track  picture. 

•  The  two  major  hardware  pieces  are  the  Cooperative 
Engagement  Processor,  which  collects  and  fuses  radar  data, 
and  the  Data  Distribution  System,  which  distributes  CEC  data 
to  other  CEC-equipped  ships  and  aircraft. 

•  CEC  is  an  integrated  component  of,  and  serves  as  the  primary 
air  tracker  for  non-LSD  class  SSDS  MK  2-equipped  ships. 

•  There  are  two  major  surface  ship  variants  of  CEC: 

-  The  CEC  AN/USG-2/2A  is  used  in  selected  Aegis  cruisers 
and  destroyers,  LPD  1 7/LHD/LHA(R)  amphibious  ships, 
and  CVN  68  class  aircraft  carriers. 
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-  The  CEC  AN/USG-2B,  an  improved  version 
of  the  AN/USG-2/2A,  is  used  in  selected  Aegis 
cruisers/destroyers  as  well  as  selected  amphibious  assault 
ships  including  the  LHA(R)  ship  class  and  CVN  68  class 
aircraft  carriers. 

SEWIP 

•  The  SEWIP  is  an  evolutionary  development  program 
providing  block  upgrades  to  the  AN/SLQ-32  Electronic 
Warfare  (EW)  System  to  address  critical  capability, 
integration,  logistics,  and  performance  deficiencies. 

•  There  are  three  major  SEWIP  block  upgrades: 

-  SEWIP  Block  1  replaced  obsolete  parts  in  the  AN/SLQ-32 
and  incorporated  a  new,  user-friendly  operator  console,  an 
improved  electronic  emitter  identification  capability,  and 
an  embedded  trainer. 

-  SEWIP  Block  2  is  in  development  and  will  incorporate 
a  new  receiver  antenna  system  intended  to  improve  the 
AN/SLQ-32’s  passive  EW  capability. 

-  SEWIP  Block  3  is  in  development  and  will  incorporate  a 
new  transmitter  antenna  system  intended  to  improve  the 
AN/SLQ-32 ’s  active  EW  capability. 

Mission 

•  Naval  Component  Commanders  use  SSDS,  RAM,  ESSM, 
SEWIP,  and  CEC,  and  many  legacy  systems  to  accomplish 
ship  self-defense  missions. 


•  Naval  surface  forces  use  the: 

-  SSDS  to  provide  automated  and  integrated 
detect-to-engage  ship  self-defense  capabilities  against 
ASCM,  air,  and  surface  threats. 

-  RAM  to  provide  a  short-range  hard-kill  engagement 
capability  against  ASCM  threats. 

-  ESSM  to  provide  a  medium-range  hard-kill  engagement 
capability  against  ASCM,  surface,  and  low-velocity  air 
threats. 

-  CEC  to  provide  accurate  air  and  surface  threat  tracking 
data  to  SSDS. 

-  SEWIP-improved  AN/SLQ-32  as  the  primary  EW 
sensor  and  weapons  system  for  air  defense  (to  include 
self-defense)  missions. 

Major  Contractors 

•  SSDS  (all  variants):  Raytheon  -  San  Diego,  California 

•  RAM  and  ESSM  (all  variants):  Raytheon  -  Tucson,  Arizona 

•  CEC  (all  variants):  Raytheon  -  St.  Petersburg,  Florida 

•  SEWIP 

-  Block  1 :  General  Dynamics  Advanced  Information 
Systems  -  Fair  Lakes,  Virginia 

-  Block  2:  Lockheed  Martin  -  Syracuse,  New  York 

-  Block  3:  Northrop  Grumman  -  Baltimore,  Maryland 


Activity 

•  COTF  conducted  at-sea  testing  for  the  LFIA(R)  IOT&E 
on  USS  America  (LFIA  6)  in  April  2015  and  the  SDTS 
from  December  2014  to  March  2015,  in  accordance  with 
DOT&E-approved  test  plans.  Completion  of  these  at-sea  test 
phases  is  scheduled  for  mid-2016. 

•  COTF  continued  planning  for  conduct  of  the  LHA(R)  PRA 
M&S  test  bed  phase  scheduled  to  commence  in  late-2016. 

Assessment 

•  The  results  of  testing  completed  to  date  indicate  that  the 
LFIA(R)  has  some  ship  self-defense  capability  against  older 
ASCM  threats.  The  LHA(R)  ship  self-defense  performance 
against  newer  ASCM  threats  will  remain  undetermined  until 
IOT&E  is  completed. 

•  Deficiencies  in  RAM  Block  2  integration  with  the  LFIA(R) 
SSDS-based  combat  system  caused  several  RAM  Block  2 
missiles  to  miss  their  targets  during  one  of  the  IOT&E  missile 
firing  scenarios  on  the  SDTS.  The  Navy  has  initiated  a  formal 
Failure  Review  Board  to  determine  the  required  corrections. 

•  SSDS  MK  2  failed  to  properly  evaluate  its  engagement 
doctrine  for  an  inbound  raid  of  ASCM  surrogates.  This 
issue  resides  within  SSDS’s  Local  Command  and  Control 
functionality  and  can  result  in  missed  engagements  against 
ASCM  threats.  The  Navy  is  still  investigating  this  issue. 


•  An  ESSM  pre-detonated  on  debris  before  approaching  its 
intended  target.  This  issue  also  occurred  during  an  earlier 
(non-LFlA(R))  ESSM  FOT&E  event.  When  this  issue  occurs, 
the  missile  will  fail  to  destroy  its  intended  target. 

•  Inactive  target  emitters  continue  to  be  reported  as  valid  by 
the  AN/SLQ-32  Electronic  Warfare  System  (EWS)  with  the 
SEWIP  Block  1  upgrade  after  the  target  was  destroyed.  These 
false  detections  contributed  to  the  SSDS  re-engaging  the 
already-destroyed  target.  This  problem  will  accelerate  the 
depletion  of  the  ship’s  missile  inventory  and  waste  combat 
system  resources  that  might  be  needed  for  engaging  other 
ASCM  threats. 

•  LFIA  6  class  ships  defend  themselves  against  ASCM  by  first 
using  the  medium-range  ESSM  and  then  the  shorter-range 
RAM.  RAM  uses  radio  frequency  and/or  infrared  terminal 
guidance  to  home  on  ASCM  threats.  Plot  debris  from  prior 
intercepts  and  warhead  detonations  can  therefore  interfere 
with  RAM’s  infrared  guidance.  While  the  SSDS  is  designed 
to  schedule  RAM  and  ESSM  engagements  to  avoid  this  type 
of  interference,  it  failed  to  do  so  during  testing. 

•  The  AN/SLQ-32  EWS  with  the  SEWIP  Block  1  upgrade 
was  not  able  to  timely  detect  certain  types  of  ASCM 
emitter  signals.  The  late  detections  negatively  affected  the 
performance  of  RAM  missiles  that  the  SSDS  employed 
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against  these  ASCM  threats.  The  Navy  is  addressing  this 
deficiency  with  the  SEWIP  Block  2  upgrade  to  the  AN/SLQ-32 
EWS. 

•  The  LHA(R)’s  AN/SPQ-9B  surface  search  radar  demonstrated 
integration  problems  with  the  ship’s  SSDS-based  combat 
system  such  that  not  all  AN/SPQ-9B  detections  were 

used  by  the  combat  system  when  tracking  targets.  Since 
the  AN/SPQ-9B  is  the  ship’s  primary  radar  for  detecting 
sea-skimming  ASCMs,  these  missed  detections  significantly 
degrade  the  combat  system’s  ability  to  schedule  self-defense 
missiles  against  this  type  of  ASCM  threat. 

•  The  LHA(R)’s  AN/SPQ-9B  surface  search  radar,  which  is 
this  ship’s  primary  radar  for  detecting  sea-skimming  ASCMs, 
demonstrated  late  detections  of  certain  types  of  ASCM  threats 
at  distances  much  closer  to  the  ship  than  predicted  from 
historical  data.  Late  detections  of  these  threats  resulted  in 
reduced  engagement  times. 

•  Two  BQM-74  aerial  targets  failed  to  maintain  operationally 
realistic  flight  parameters  in  one  of  the  IOT&E  missile  firing 
scenarios. 

•  Due  to  the  lack  of  a  Multi-Stage  Supersonic  Target  (MSST), 
no  assessment  of  the  LHA(R)  ship  self-defense  capability 
against  MSST-like  ASCM  threats  is  possible. 

•  Three  of  four  ESSM  missiles  failed  to  defeat  their  assigned 
targets  during  the  two  missile  firing  exercises  on  USS 
America.  The  Navy  is  investigating  the  causes  of  these 
failures. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy 
has  satisfactorily  completed  some  of  the  previous 
recommendations.  However,  the  Navy  has  not  resolved  the 
following  previous  recommendations  related  to  LHA(R)  ship 
self-defense: 

1.  Optimize  SSDS  MK  2  weapon  employment  timelines  to 
maximize  weapon  Probability  of  Kill. 

2.  Develop  a  credible  open-loop  seeker  subsonic  ASCM 
surrogate  target  for  ship  self-defense  combat  system 
operational  tests. 

3.  Correct  the  identified  SSDS  MK  2  software  reliability 
deficiencies. 

4.  Correct  the  identified  SSDS  MK  2  training  deficiencies. 

5.  Develop  and  field  deferred  SSDS  MK  2  interfaces  to  the 
Global  Command  and  Control  System  -  Maritime  and  the 
TPX-42A(V)  command  and  control  systems. 

6.  Continue  to  implement  the  Program  Executive  Office 
for  Integrated  Warfare  Systems’  plan  for  more  robust, 
end-to-end  systems  engineering  and  associated 
developmental/operational  testing  of  ship  self-defense 
combat  systems. 

7.  Improve  the  ability  of  legacy  ship  self-defense  combat 
system  sensor  elements  to  detect  threat  surrogates  used  in 
specific  ASCM  raid  types. 

8.  Develop  adequate  and  credible  target  resources  for  ship 
self-defense  and  EW  operational  testing. 


9.  Continue  to  take  action  on  the  classified  recommendations 
contained  in  the  March  2011  and  November  2012  DOT&E 
reports  to  Congress  on  the  ship  self-defense  mission  area. 

10.  Improve  the  SSDS  MK  2  integration  with  the  MK  9  Track 
Illuminators  to  better  support  ESSM  engagements. 

11.  Develop  combat  system  improvements  to  increase  the 
likelihood  that  ESSM  and  RAM  will  home  on  their 
intended  targets. 

12.  Continue  to  implement  and  demonstrate  with  adequate 
operational  testing  the  ship  self-defense  Fire  Control  Loop 
Improvement  Program  improvements. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Correct  the  identified  integration  deficiencies  with  the 
LHA(R)  SSDS-based  combat  system  and  the  RAM  Block 
2  missile.  Demonstrate  these  corrections  in  a  phase  of 
LHA(R)  operational  testing. 

2.  Correct  the  cause  of  the  ESSM  missile  failures  and 
demonstrate  the  correction  in  a  future  phase  of  operational 
testing. 

3.  Correct  the  SSDS  Local  Command  and  Control 
functionality  issue.  Demonstrate  the  correction  in  a  future 
phase  of  SSDS  operational  testing. 

4.  Investigate  means  to  mitigate  the  chances  of  an  ESSM 
pre-detonating  on  debris  before  approaching  its  intended 
target. 

5.  Investigate  why  target  emitters  continue  to  be  reported  as 
valid  by  the  AN/SLQ-32  EWS  with  the  SEWIP  Block  1 
upgrade  after  the  target  is  destroyed.  Test  any  corrections  in 
a  future  operational  test  phase. 

6.  Correct  the  SSDS  scheduling  function  to  preclude 
interference  from  prior  intercepts  and  warhead  detonations 
with  RAM’s  infrared  guidance.  Demonstrate  corrections  in 
a  phase  of  operational  testing. 

7.  Correct  the  integration  problems  with  the  SSDS-based 
combat  system  and  the  AN/SPQ-9B  radar  to  ensure  that  all 
valid  AN/SPQ-9B  detections  are  used  by  the  combat  system 
when  tracking  targets.  Demonstrate  the  corrections  in  a 
phase  of  operational  testing. 

8.  Investigate  the  late  detections  by  the  AN/SPQ-9B  radar 
of  certain  types  of  threats  at  distances  much  closer  to  the 
ship  than  predicted  from  historical  data.  Demonstrate  any 
corrections  in  a  future  phase  of  operational  testing. 

9.  Investigate  why  the  BQM-74  aerial  targets  failed  to 
maintain  operationally  realistic  flight  parameters. 
Demonstrate  any  corrections  prior  to  using  these  targets  in 
similar  operational  test  scenarios. 

10.  Complete  the  LHA(R)  IOT&E  at-sea  test  phases  and  the 
planning  for  the  LHA(R)  PRA  M&S  test  bed  IOT&E  test 
phase. 

11.  Update  the  LHA(R)  Test  and  Evaluation  Master  Plan  to 
include  at-sea  and  PRA  test  bed  operational  test  phases  to 
enable  evaluation  of  ship  self-defense  capabilities  on  the 
LHA-8  equipped  with  new  radar  systems. 
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Ship-to-Shore  Connector  (SSC) 


Executive  Summary 

•  This  is  the  first  annual  report  for  this  program  and  it  only 
addresses  LFT&E. 

•  In  June  and  July  2015,  the  Navy  conducted  full  scale  hull  tests 
of  the  Ship-to-Shore  Connector  (SSC)  hull  and  skirt  system 
against  an  under-hull  explosion  and  blast/ fragmentation  threat 
using  a  Landing  Craft  Air  Cushion  (LCAC)  as  a  surrogate. 

The  full  hull  tests  generated  data  that  will  be  used  to  evaluate 
the  potential  for  crew  and  troop  casualties  and  characterize  the 
weapons  effects  on  craft  structure  and  machinery. 

•  The  Navy  tested  the  topside  modules  and  structure  of  the 
LCAC,  the  system  the  SSC  is  replacing,  against  weapons 
effects  from  shaped  charge,  indirect  fire,  and  land  mine  threats 
during  its  development  in  2011  and  2012.  The  SSC  Program 
Office  used  the  results  of  these  tests  to  identify  knowledge 
gaps  for  the  SSC  and  accordingly  fund  and  plan  additional 
testing  to  address  these  gaps. 

•  The  first  craft  is  scheduled  for  delivery  in  FY17,  and  initial 
operational  capability  for  the  SSC  is  projected  to  occur  in 
FY20. 

System 

•  The  SSC  is  a  fully  amphibious  air  cushion  vehicle  intended  to 
replace  the  existing  LCACs. 

•  Compared  to  the  existing  LCAC,  the  Navy  intends  the  SSC  to 
have  increased  payload,  reliability,  and  availability. 

•  The  Navy  intends  to  operate  the  SSC  from  the  well  decks  of 
current  and  planned  Navy  amphibious  ships  and  onboard  the 
planned  Mobile  Landing  Platform. 


Mission 

Commanders  will  employ  amphibious  crews  equipped  with  the 
SSC  to  transport  equipment,  personnel,  and  weapons  systems 
from  ships  through  the  surf  zone  and  across  the  beach  to  landing 
points  beyond  the  high  water  mark  in  a  variety  of  environmental 
conditions. 

Major  Contractor 

Textron  Systems  -  New  Orleans,  Louisiana 


Activity 

•  In  201 1  and  2012,  sections  of  an  LCAC  structure  and  topside 
modules  were  used  to  characterize  the  weapons  effects  from 
shaped  charge,  indirect  fire,  and  land  mine  threats  when  the 
craft  was  on  cushion.  Data  from  these  tests  were  used  to 
develop  the  full  hull  test  plan. 

•  In  July  2013,  the  Navy  conducted  a  test  using  a  1/10  scale 
SSC  in  a  wave  tank  to  evaluate  the  craft’s  ability  to  survive  a 
10-foot  significant  wave  height  sea  condition.  The  scalability 
of  these  data  will  be  validated  by  instrumented  trials  using  the 
first  SSC  after  delivery  from  the  shipbuilder  in  FY17. 

•  In  June  and  July  2015,  the  Navy  completed  full  hull  testing 
using  an  operational  LCAC  as  a  surrogate  for  the  SSC.  This 
included  two  tests  against  mines  emplaced  under  the  hull  and 
skirt,  as  well  as  a  full  hull  test  against  a  blast/fragmentation 
threat.  The  Navy  intends  to  install  energy-attenuating  seats  in 
the  Command  Module  of  the  SSC  to  increase  ride  comfort  of 
the  crew,  but  these  seats  were  not  available  for  the  under-hull 
test. 


•  Armor  characterization  is  scheduled  for  early  FY16,  pending 
procurement  of  the  armor  that  meets  the  SSC  specifications. 

•  The  Navy  conducted  all  testing  in  accordance  with  a 
DOT&E-approved  test  plan. 

Assessment 

•  The  LFT&E  conducted  to  date  have  provided  data  that  can 
be  used  to  refine  craft  damage  predictions  and  crew  and  troop 
casualty  predictions.  These  data  should  be  incorporated  into 
the  modeling  and  simulation  of  the  SSC  and  the  engineering 
assessments  that  will  be  utilized  in  the  Navy’s  SSC 
survivability  assessment. 

•  The  energy-attenuating  seats  were  unavailable  for  installation 
on  the  full  hull  test,  but  the  Navy  collected  data  to  facilitate 
future  analyses  to  evaluate  the  performance  of  the  new  seats, 
when  they  become  available. 
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•  Findings  from  the  completed  full  hull  testing  are  based  on 
using  LCAC  as  a  surrogate.  The  Navy  should  complete  an 
engineering  assessment  to  determine  if  additional  testing  to 
examine  SSC-specific  components  is  required. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Evaluate  the  classified  findings  from  the  full  hull  test 
to  determine  if  the  risk  for  personnel  casualties  can  be 
reduced. 


2.  Evaluate  the  results  of  the  full  hull  test  to  determine  if  the 
LCAC  component  performance  following  the  weapons 
effects  presented  is  adequately  representing  the  predicted 
performance  of  SSC  components  and/or  to  determine  if 
additional  SSC-specific  component  testing  is  warranted. 

3.  Test  the  energy-absorbing  seat  design  for  the  pilot  and 
co-pilot  at  the  structural  load  levels  measured  during  the 
full  hull  test.  These  seats  were  not  available  for  installation 
at  the  time  of  the  test. 
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Small  Surface  Combatant  (SSC) 
Modification  to  the  Littoral  Combat  Ship  (LCS) 


Executive  Summary 

•  In  December  2014,  the  SECDEF  approved  the  Navy’s 
recommendation  to  limit  the  Small  Surface  Combatant 
(SSC)  requirements  to  what  could  be  accomplished  within 
cost  constraints  using  a  modification  to  the  existing  Littoral 
Combat  Ship  (LCS)  Flight  0+  baseline  configurations. 

•  In  August  and  October  2015,  the  Navy  delivered  two  drafts 
of  the  Capability  Design  Documents  (CDD)  that  relegate  all 
mission  performance  measures,  other  than  the  two  measures 
for  force  protection  against  surface  and  air  threats,  to  Key 
System  Attributes  rather  than  Key  Performance  Parameters 
(KPPs),  which  permits  the  combat  capabilities  desired  in  these 
follow-on  ships  to  be  traded  away  as  needed  to  remain  within 
the  cost  constraints.  As  a  residt,  the  new  SSC  could,  in  the 
extreme,  be  delivered  with  less  mission  capability  than  desired 
and  with  limited  improvements  to  the  survivability  of  the  ship 
in  a  combat  environment.  In  fact,  the  SSC  could  meet  all  its 
KPPs  without  having  any  mission  capability. 

System 

•  In  2014,  the  SECDEF  authorized  the  Navy  to  restructure 
the  LCS  program  to  build  the  final  20  ships  in  the  program 
(ships  33  through  52)  to  a  revised  version  of  the  LCS  Flight 
0+  baseline  design.  The  revised  design,  potentially  starting 
in  FY19,  is  designated  the  SSC;  the  Navy  is  now  also  calling 
it  a  Fast  Frigate.  Some  SSC  components  and  design  changes 
yet  to  be  identified  may  also  be  incorporated  into  LCS  hulls 
25  through  32  (FY16  -  FY18). 

•  The  revised  design  that  the  Navy  wants  to  use  for  the  SSC 
includes  additional  or  improved  built-in  equipment  for 
Surface  Warfare  (SUW),  Anti-Submarine  Warfare  (ASW),  and 
Anti-Air  Warfare.  These  ships  are  expected  to  be  heavier  than 
the  Flight  0+  baseline  LCS  design  and  have  a  lower  maximum 
sustained  speed  and  un-refueled  range.  They  would  retain 
some  modularity  to  enable  them  to  be  configured  for  either  full 
SUW  or  ASW  missions  by  swapping  portions  of  the  mission 
modules: 

-  From  the  SUW  mission  package  ,  the  following  would  be 

removed  to  convert  to  full  ASW  mission  capability: 

■  30  mm  guns 

■  Ship-launched  (Longbow)  HELLFIRE  Missile 

■  11m  boats 

-  From  the  ASW  mission  package  ,  the  following  would  be 

removed  to  convert  to  full  SUW  mission  capability: 

■  Variable  Depth  Sonar  (active  transmitter) 

■  ASW  Engagement  Weapons  for  helicopter 

•  The  SSC  will  not  be  able  to  perform  the  mine  countermeasure 
mission. 


•  While  the  Navy  has  not  released  a  final  design,  it  is 
considering  the  following  modifications  to  the  LCS  Flight  0+ 
baseline: 

-  An  improved  three-dimensional  air  surveillance  radar 

-  An  upgrade  of  the  ship’s  air  defense  capability  to  include 
Sea  Rolling  Airframe  Missile  Anti-Ship  Missile  Defense 
System  (already  part  of  the  Independence  variant 
seaframe) 

-  An  over-the-horizon  SUW  anti-ship  missile 

-  An  improved  electronic  warfare  capability 

-  Improved  decoy  systems  for  air  defense 

-  A  multifunction  towed-array  passive  sonar  system 

-  Torpedo  defense  and  countermeasures  equipment 

-  Increased  magazine  armor 

-  25  mm  guns 

•  The  Navy  has  not  made  a  decision  on  the  SSC  seaframe. 
Currently,  two  variants  are  produced: 

-  The  Freedom  variant  is  a  semi-planing  monohull  design 
constructed  of  steel  (hull)  and  aluminum  (deckhouse)  with 
two  steerable  and  two  fixed-boost  water  jets  driven  by  a 
combined  diesel  and  gas  turbine  main  propulsion  system. 

-  The  Independence  variant  is  an  aluminum  trimaran  design 
with  two  steerable  water  jets  driven  by  diesel  engines  and 
two  steerable  water  jets  driven  by  gas  turbine  engines. 

Mission 

•  The  Maritime  Component  Commander  will  employ  SSC 
to  conduct  ASW  or  SUW  tasks  depending  on  the  mission 
components  fitted  into  the  seaframe.  Commanders  can 
employ  LCS  in  a  maritime  presence  role  in  any  configuration 
to  build  and  strengthen  maritime  partnerships  by  training  and 
operating  with  smaller,  regional  navies. 

•  The  Navy  has  not  yet  published  a  Concept  of  Operations  for 
the  SSC,  but  the  Navy  reported  in  its  2014  SSC  assessment 


SSC  LCS  293 


F Y1  5  NAVY  PROGRAMS 


report  that  the  SSC  would  operate  as  an  ASW  or  SUW  escort 
for  high-value  units,  in  a  surface  action  group  focused  on  ASW 
or  SUW,  or  operate  independently  ahead  of  the  strike  group 
preparing  the  operational  environment  for  joint  force  assured 
access  to  critical  littoral  regions.  The  level  of  Probability  of 
Raid  Annihilation  requirement  indicated  in  the  draft  CDD 
implies  the  SSC  is  likely  to  operate  under  an  air  defense 
umbrella  provided  by  other  units  as  determined  necessary  by 
the  operational  commander. 

Major  Contractors 

An  acquisition  strategy  for  the  20  SSC  has  not  been  approved  and 
no  contracts  have  been  authorized.  The  current  LCS  production 
lines  are: 


•  Freedom  class  variant  (mono-hull  design) 

-  Prime:  Lockheed  Martin  Maritime  Systems  and 
Sensors  -  Washington,  District  of  Columbia 

-  Shipbuilder:  Marinette  Marine  -  Marinette,  Wisconsin 

•  Independence  class  variant  (trimaran  design) 

-  Prime:  Austal  USA  -  Mobile,  Alabama 

-  Shipbuilder:  Austal  USA- Mobile,  Alabama 

•  Mission  Packages 

-  Future  Mission  Package  Integration  contract  awarded  to 
Northrop  Grumman  -  Los  Angeles,  California 


Activity 

•  In  November  2014,  in  response  to  the  SECDEF’s 
February  2014  memorandum  tasking  the  Navy  to  examine 
the  needs  of  the  fleet  and  propose  alternate  designs  to  procure 
a  small  combatant  with  capabilities  consistent  with  modern 
frigates,  the  Navy  recommended  a  SSC  design  based  on  minor 
modification  to  the  LCS. 

•  In  December  2014,  the  SECDEF  directed  the  Navy  to  move 
forward  with  a  multi-mission  SSC  based  on  the  existing  LCS 
Flight  0+  baseline  configurations. 

•  The  Navy  is  still  in  the  process  of  developing  the  SSC 
acquisition  strategy,  the  detailed  ship  designs,  and  selecting  the 
systems  and  components  for  this  modification. 

•  During  2-4QFY15,  the  Navy  developed  a  draft  CDD,  which  is 
currently  under  review  but  expected  to  be  approved  in  FY 16. 

•  In  August  and  October  2015,  DOT &E  reviewed  two  versions 
of  the  draft  CDD  and  provided  critical  comments  on  the 
document  to  the  Joint  Staff  and  the  Navy. 

Assessment 

•  The  latest  draft  SSC  CDD  requires  that  the  modified  LCS 
be  multi-mission  capable,  more  lethal,  and  more  survivable. 
The  SSC  is  required  to  have  mission  system  components 
from  the  LCS  SUW  and  ASW  mission  modules  to  allow  the 
ships  to  conduct  some  degree  of  the  SUW  and  ASW  missions 
simultaneously.  Additionally,  the  draft  SSC  CDD  cites  that 
based  on  cost,  schedule,  and  performance,  components  of  an 
LCS  mission  package  may  be  installed  on  a  full  time  basis 
for  space,  weight,  power,  and  cooling  (SWaP-C)  savings. 
However,  because  of  SWaP-C  limitations  inherent  in  the 
current  LCS  design,  the  SSC  most  likely  will  not  meet  all  of 
the  requirements  specified  in  the  draft  CDD  simultaneously,  it 
will  most  likely  require  swapping  mission  modules  to  provide 
either  the  full  mission  capability  for  SUW  or  ASW  as  directed 
by  the  Force  Commander. 

•  The  latest  draft  CDD  relegates  all  mission  performance 
measures,  other  than  the  two  measures  for  force  protection 
against  surface  and  air  threats,  to  Key  System  Attributes  rather 
than  KPPs,  which  permits  the  combat  capabilities  desired  in 


these  follow-on  ships  to  be  traded  away  as  needed  to  remain 
within  the  cost  constraints.  As  a  result,  the  new  SSC  could, 
in  the  extreme,  be  delivered  with  less  mission  capability  than 
desired  and  with  limited  improvements  to  the  survivability  of 
the  ship  in  a  combat  environment.  In  fact,  the  SSC  could  meet 
all  its  KPPs  without  having  any  mission  capability. 

•  The  vulnerability  reduction  features  proposed  for  the  SSC 
provide  no  significant  improvement  in  the  ship’s  survivability. 
Notwithstanding  potential  reductions  to  its  susceptibility  due 
to  improved  electronic  warfare  system  and  torpedo  defense, 
minor  modifications  to  LCS  (e.g.,  magazine  armoring)  will 
not  yield  a  ship  that  is  significantly  more  survivable  than  LCS 
when  engaged  with  threat  missiles,  torpedoes,  and  mines 
expected  in  major  combat  operations. 

•  The  current  LCS  seaframes  do  not  have  sufficient  separation 
and  redundancy  in  their  vital  systems  to  recover  damaged 
capability.  Because  the  SSC  design  is  not  substantially 
different  from  the  LCS  Flight  0+  baseline  and  will  not  add 
much  more  redundancy  or  greater  separation  of  critical 
equipment  or  additional  compartmentation,  it  will  likely  be 
less  survivable  than  the  Navy’s  previous  frigate  class. 

•  While  the  Navy  is  examining  methods  to  reduce  weight,  it 
is  anticipated  the  SSC  will  be  heavier  than  the  existing  LCS 
resulting  in  a  lower  maximum  sprint  speed  and  less  fuel 
endurance. 

Recommendations 

•  Status  of  Previous  Recommendations.  This  is  the  first  annual 
report  for  this  program. 

•  FY  15  Recommendations.  The  Navy  should: 

1 .  Improve  the  final  CDD  by  developing  clearly-defined 
mission-focused  requirements  for  SUW,  ASW,  and  Air 
Warfare,  and  specifying  them  as  KPPs  for  each  focused 
mission  configuration. 

2.  Consider  implementing  additional  survivability 
improvement  measures  (e.g.,  shock  hardening,  redundancy 
of  vital  systems,  etc.)  to  make  SSC  more  survivable  in 
combat. 
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SSN  774  Virginia  Class  Submarine 


Executive  Summary 

•  The  Navy  deployed  die  first  Virginia  class  Block  III 
submarine,  USS  North  Dakota  (SSN  784),  in  May  2015 
with  only  limited  developmental  testing  of  the  platform’s 
major  subsystem  upgrades.  Major  testing  phases  included 
developmental  testing  of  the  new  Large  Aperture  Bow  (LAB) 
sonar  array,  testing  of  the  system  to  support  weapon  system 
accuracy  (this  included  sonar  performance  assessments), 
testing  of  the  weapon  system  interfaces,  and  a  limited 
operational  assessment  phase  to  support  deployment 
certification. 

•  DOT&E  submitted  a  classified  Early  Fielding  Report  in 
September  2015  detailing  the  results  of  the  testing  to  date. 
DOT&E  concluded  that: 

-  The  Virginia  class  Block  III  submarine  with  the  LAB  array 
has  the  potential  to  perform  as  an  adequate  replacement  for 
the  spherical  array  used  on  previous  Virginia  class  variants. 

-  System  reliability  meets  the  Navy’s  thresholds. 

-  The  new  LAB  array  and  the  Light  Weight  Wide  Aperture 
Array  (LWWAA)  sonar  processing  systems  suffer  from 
some  deficiencies.  Although  the  Navy  has  implemented 
corrective  action  in  each  case,  a  full  operational  evaluation 
has  not  yet  been  conducted. 

•  The  Navy  intends  to  conduct  a  comprehensive  operational 
test  of  the  Block  III  system  starting  in  1QFY17  covering 
the  Anti-Submarine  Warfare  (ASW),  Anti-Surface  Warfare 
(ASUW),  Strike  Warfare,  and  Intelligence  collection  mission 
areas  in  addition  to  suitability  and  cybersecurity  assessments. 

•  The  Navy  has  conducted  four  major  test  periods  to  assess  the 
Virginia  class  Block  I  and  Block  II  submarines.  The  overall 
results  of  these  combined  testing  periods  were: 

-  Virginia  class  submarines  are  capable  of  hosting  the  Dry 
Deck  Shelter  system  and  can  remain  covert  during  Naval 
Special  Warfare  missions  in  some  environments  against 
some  threat  forces. 

-  Virginia  class  submarines  are  effective  at  supporting 
general  arctic  operations. 

-  Virginia  class  submarines  are  effective  at  conducting  ASW 
missions  in  some  environments  against  some  threats. 

-  Although  the  Virginia  class  submarine  was  not  effective  for 
some  missions  tested,  it  remains  an  effective  replacement 
for  the  Los  Angeles  class  submarine,  providing  similar 
mission  performance  and  improved  covertness. 

System 

•  The  Virginia  class  submarine  is  the  Navy’s  latest  fast 
attack  submarine  that  is  capable  of  targeting,  controlling, 
and  launching  MK  48  Advanced  Capability  torpedoes  and 
Tomahawk  cruise  missiles. 


•  The  Navy  is  procuring  Virginia  class  submarines  incrementally 
in  a  series  of  blocks;  the  block  strategy  is  for  contracting 
purposes,  not  necessarily  to  support  upgrading  capabilities. 

-  Block  I  (hulls  1-4)  and  Block  II  (hulls  5-10)  ships  were 
built  to  the  initial  design  of  the  Virginia  class. 

-  Block  III  (hulls  11-18)  and  Block  IV  (hulls  19-28)  ships 
include  the  following  affordability  enhancements  starting 
with  SSN  784,  USS  North  Dakota : 

■  A  LAB  array  in  place  of  the  spherical  array  in  the  front 
of  the  ship. 

■  Two  Virginia  payload  tubes  replace  the  12  vertical 
launch  tubes;  each  payload  tube  is  capable  of  storing  and 
launching  six  Tomahawk  land  attack  missiles  used  in 
Strike  Warfare  missions. 

-  Block  V  and  beyond  will  increase  strike  payload  capacity 
from  12  to  40  Tomahawk  land  attack  missiles  by  adding 
a  set  of  four  additional  payload  tubes  in  an  amidships 
payload  module,  capable  of  storing  and  launching  seven 
Tomahawk  missiles  each,  as  well  as  providing  the  potential 
to  host  future  weapons  and  unmanned  systems. 

Mission 

The  Operational  Commander  will  employ  the  Virginia  class 
submarine  to  conduct  open-ocean  and  littoral  covert  operations 
that  support  the  following  submarine  mission  areas: 

•  Strike  Warfare 

•  Anti-Submarine  Warfare 

•  Intelligence,  Surveillance,  and  Reconnaissance 

•  Mine  Warfare 

•  Anti- Surface  Ship  Warfare 

•  Naval  Special  Warfare 

•  Battle  Group  Operations 


SSN  784  Virginia 


295 


F Y1  5  NAVY  PROGRAMS 


Major  Contractors 

•  General  Dynamics  Electric  Boat  -  Groton,  Connecticut 

•  Huntington  Ingalls  Industries,  Newport  News 
Shipbuilding  -  Newport  News,  Virginia 


Activity 

•  The  Navy  completed  the  shock  qualification  testing  for  the 
Virginia  Common  Weapons  Launcher  and  the  Virginia  Payload 
Tube  hatch  in  late  2014.  The  analysis  of  the  test  results 

and  redesign  of  a  sub  component  has  resulted  in  the  hatch 
qualification  being  delayed  until  late  2015. 

•  The  Navy  has  delayed  the  validation  of  the  Transient 
Shock  Analysis  modeling  method  used  for  the  design  of 
Virginia  class  Block  III  items  until  mid  to  late  2016  to  allow 
more  time  to  complete  their  analysis. 

•  The  update  of  the  Virginia  Class  Vulnerability  Assessment 
report,  which  will  include  the  Block  III  modifications,  will  be 
delayed  to  late  2016. 

•  The  following  developmental  testing  events  were  completed 
prior  to  deploying  the  first  Virginia  class  Block  III  platform 
with  data  from  each  event  provided  to  DOT&E  for  analysis: 

-  In  September  2010,  the  Navy  completed  a  full  scale  LAB 
array  acoustic  and  vibration  developmental  test. 

-  In  October  2014,  the  Navy  completed  the  first  Block  III 
Virginia  class  submarine  Weapon  System  Accuracy  Test, 
which  assessed  the  overall  systems  ability  to  employ 
weapons. 

-  In  December  2014,  the  Navy  conducted  a  limited  free  play 
test  event  to  support  an  early  system  follow-on  assessment 
to  re-evaluate  the  LAB  array  sonar. 

•  In  May  2015,  the  Navy  conducted  a  limited  free  play 
developmental  test  event  observed  by  DOT&E  to  evaluate 
the  LAB  array  in  support  of  an  early  deployment.  In 
September  2015,  DOT&E  submitted  a  classified  Early 
Fielding  Report  on  the  first  Virginia  class  Block  III  submarine 
in  response  to  an  early  deployment  prior  to  the  completion  of 
developmental  or  operational  testing. 

•  The  Navy  is  planning  to  conduct  a  comprehensive  operational 
test  of  the  Block  III  system  in  early  FY17  covering  ASW, 
ASUW,  Strike  Warfare,  and  Intelligence  collection  mission 
areas,  in  addition  to  suitability  and  cybersecurity  assessments. 

•  Due  to  material  issues  associated  with  Electric  Boat 
sub-vendor  work,  all  Virginia  class  Block  III  operational 
testing  previously  scheduled  for  completion  in  3QFY16  has 
been  delayed  until  2QFY17. 

Assessment 

•  The  September  2015  DOT&E  classified  Early  Fielding 
Report  details  the  impact  of  the  new  major  components  of  the 
system  with  respect  to  the  intended  mission  during  the  early 
deployment.  The  report  concluded  the  following: 

-  The  changes  to  the  Virginia  class  Block  III  submarine  do 
not  appear  to  improve  or  degrade  the  system’s  ability  to 
conduct  submarine  missions. 
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-  The  LAB  array  demonstrates  the  potential  to  perform  as  an 
adequate  replacement  for  the  legacy  spherical  array. 

-  Although  the  technical  parameters  are  similar,  the  system 
initially  presented  a  series  of  display  artifacts,  which 
could  affect  performance.  The  Navy  has  investigated 
these  artifacts  and  has  issued  software  fixes  to  mitigate  the 
effects.  This  software  has  not  been  operationally  tested. 

-  The  sonar  LWWAA  experienced  a  hardware  fault,  which 
limited  the  effectiveness  of  the  system.  The  Navy 
investigated  the  issue,  determined  the  root  cause,  and 
implemented  a  software  update  to  help  mitigate  the  issue. 

-  Developmental  testing  of  the  system  indicates  that  system 
software  reliability  meets  the  Navy’s  thresholds.  Hardware 
reliability  was  not  able  to  be  evaluated  because  of  the 
limited  time  available  to  testers  for  the  evaluation.  The 
LAB  array  outboard  signal  processing  equipment  has 
exhibited  some  early  failures.  The  Navy  issued  fleet 
guidance  for  monitoring  system  performance  and  continues 
to  investigate  potential  causes. 

Recommendations 

•  Status  of  Previous  Recommendations. 

-  The  Navy  has  made  progress  in  addressing  25  of  the 
44  recommendations  contained  in  the  November  2009 
and  2012  classified  FOT&E  reports.  However,  there 
are  several  that  remain  outstanding.  The  significant 
unclassified  recommendations  are: 

1 .  Test  against  a  diesel  submarine  threat  surrogate  in 
order  to  evaluate  Virginia ’s  capability,  detectability,  and 
survivability  against  modern  diesel-electric  submarines. 

2.  Conduct  an  FOT&E  to  examine  Virginia ’s  susceptibility  to 
airborne  ASW  threats  such  as  Maritime  Patrol  Aircraft  and 
helicopters. 

-  The  following  recommendations  from  the  FY12  Annual 
Report  remain  open  and  the  Navy  should  work  to  address 
them  in  the  upcoming  fiscal  year: 

1.  Coordinate  the  Virginia,  A-RCT,  andAN/BYG-1  Test  and 
Evaluation  Master  Plans  and  utilize  Undersea  Enterprise 
Capstone  documents  to  facilitate  testing  efficiencies. 

2.  Complete  the  verification,  validation,  and  accreditation  of 
the  Transient  Shock  Analysis  method  used  for  Virginia  class 
Block  III  items. 

3.  Repeat  the  FOT&E  event  to  determine  Virginia  s 
susceptibility  to  low-frequency  active  sonar  and  the 
submarine’s  ability  to  conduct  ASUW  in  a  low-frequency 
active  environment.  This  testing  should  include  a 

Los  Angeles  class  submarine  operating  in  the  same 
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environment  to  enable  comparison  with  the  Virginia  class 
submarine. 

-  The  following  recommendations  from  the  FY13 

Annual  Report  remain  open  and  the  Navy  should  work  to 
address  them  in  the  upcoming  fiscal  year: 

1.  Reconsider  the  metrics  used  to  assess  the  Virginia  class 
submarine’s  ability  to  covertly  conduct  mass  swimmer 
lockout  operations  using  the  Dry  Deck  Shelter. 

2.  Investigate  and  implement  methods  to  aid  the  Special 
Operation  Forces  in  identifying  the  submarine  during 
operations  in  conditions  of  low  visibility. 

3.  Investigate  modifying  the  reducer  in  the  air  charging  system 
to  allow  higher  air  pressure  for  the  Seal  Delivery  Vehicle 
(SDV)  Auxiliary  Life  Support  System  in  order  to  provide 


increased  flexibility  for  SDV  missions  that  can  be  hosted 
from  Virginia  class  submarines. 

4.  Continue  to  collect  data  on  the  susceptibility  of  the 
Virginia  class  to  low-frequency  passive  systems  and 
conduct  a  more  quantitative  assessment  (e.g.,  determine 
detection  ranges  for  different  ship  postures). 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Address  the  three  classified  recommendations  listed  in  the 
September  2015  Block  III  Virginia  class  Early  Fielding 
Report. 

2.  Complete  operational  testing  of  the  Block  III  Virginia  class 
submarine  as  soon  as  practical. 
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Standard  Missile-6  (SM-6) 


Executive  Summary 

•  The  performance  deficiency  discovered  during  IOT&E  and 
outlined  in  the  classified  Standard  Missile-6  (SM-6)  IOT&E 
report  of  May  2013  remains  unresolved  and  continues  to  affect 
DOT&E’s  final  assessment  of  effectiveness. 

-  The  Navy  is  assessing  several  options  for  a  solution,  each 
with  varying  degrees  of  complexity.  A  primary  concern  is 
to  ensure  the  solution  causes  no  degradation  to  the  existing 
SM-6  performance  envelope. 

-  The  Navy  plans  to  incorporate  these  changes  in  Block  I 
(BLK I)  and  Block  1A  (BLK  1A)  production  variants  in 
FY16. 

•  Upon  completion  of  the  current  phase  of  SM-6  FOT&E, 
the  Navy  will  have  conducted  testing  that  will  allow  an 
assessment  of  the  SM-6  Capability  Production  Document 
performance  requirement  for  interoperability. 

•  In  FY16,  the  Navy  expects  to  demonstrate  the  maximum  range 
Key  Performance  Parameter  (KPP)  during  SM-6  FOT&E 

and  Aegis  Baseline  9  operational  testing  as  well  as  the  launch 
availability  KPP. 

•  The  Navy  commenced  developmental  testing  of  pre-planned 
product  improvements  to  the  SM-6  BLK  I  missile  in  FY14; 
these  improvements  are  the  SM-6  BLK  IA  configuration.  A 
successful,  pre-production  developmental  flight  test  (Guidance 
Test  Vehicle-1  (GTV-1))  occurred  in  FY14.  The  Navy 
successfully  conducted  a  second  GTV  mission  (GTV-2)  in 
FY15.  The  Navy  plans  to  conduct  a  final  SM-6  BLK  1A  GTV 
mission  (GTV-3)  in  FY16.  Operational  testing  of  the  SM-6 
BLK  I A  is  planned  for  FY16/17. 

•  The  Navy  conducted  seven  SM-6  BLK  1  missile  tests  during 
FY15.  Of  the  planned  launches,  two  of  three  successfully 
supported  FOT&E  with  Aegis  Baseline  9;  one  test  resulted 
in  a  missile  failure-to-launch  (dud/misfire);  one  successfully 
supported  Naval  Integrated  Fire  Control  -  Counter  Air 
(NIFC-CA)  From-the-Sea  (FTS)  Increment  1  capability;  and 
three  successfully  supported  Missile  Defense  Agency  (MDA) 
Sea-Based  Terminal  (SBT)  testing. 

•  NIFC-CA  FTS  Increment  1  test  events  have  demonstrated 
a  basic  capability,  but  its  effectiveness  under  operationally 
realistic  conditions  is  undetermined. 

•  DOT&E  continues  to  monitor  the  uplink/downlink  antenna 
shroud  reliability  during  FOT&E.  There  are  no  recorded 
failures  in  testing  since  IOT&E  in  FY11. 

System 

•  SM-6  is  the  latest  evolution  of  the  Standard  Missile  family  of 
fleet  air  defense  missiles. 

•  SM-6  is  employed  from  cruisers  and  destroyers  equipped  with 
the  Aegis  combat  systems. 


•  The  SM-6  seeker  and  terminal  guidance  electronics  derive 
from  technology  developed  in  the  Advanced  Medium-Range 
Air-to-Air  Missile  program. 

•  SM-6  retains  the  legacy  Standard  Missile  semi-active  radar 
homing  capability. 

•  SM-6  receives  midcourse  flight  control  from  the  Aegis 
Combat  System  via  ship’s  radar;  terminal  flight  control  is 
autonomous  via  the  missile’s  active  seeker  or  supported  by  the 
Aegis  Combat  System  via  the  ship’s  illuminator. 

•  SM-6  is  being  upgraded  to  the  BLK  IA  configuration  to 
address  hardware  and  software  improvements  and  to  address 
advanced  threats. 

•  SM-6  Dual  I  capability  is  being  added  to  provide  SBT 
capability  against  short-range  ballistic  missiles. 

Mission 

•  The  Joint  Force  Commander/Strike  Group  Commander  will 
use  SM-6  for  air  defense  against  fixed-/rotary- winged  targets 
and  anti-ship  missiles  operating  at  altitudes  ranging  from  very 
high  to  sea  skimming. 

•  The  Joint  Force  Commander  will  use  SM-6  as  part  of  the 
NIFC-CA  FTS  operational  concept  to  provide  extended  range 
over-the-horizon  capability  against  at-sea  and  overland  threats. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 
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Activity 

•  The  Navy  conducted  seven  SM-6  BLK I  missile  tests 
and  one  SM-6  BLK  1A  missile  test  during  FY15.  Of  the 
planned  launches,  two  of  three  successfully  supported 
FOT&E  with  Aegis  Baseline  9;  one  test  resulted  in  a  missile 
failure-to-launch  (dud/misfire);  one  successfully  supported 
N1FC-CAFTS  Increment  1  capability;  and  three  SM-6  Dual  I 
missiles  successfully  supported  MDA  SBT  and  Air  Warfare 
retention  capability.  The  single  SM-6  BLK  IA  was  success  till. 
SM-6  BLK  I  FOT&E 

•  In  March  2015,  at  Point  Mugu,  California: 

-  An  SM-6  BLK  I  FOT&E  mission  (Dll)  successfully 
engaged  a  target  that  was  using  electronic  attack  against 
the  SM-6  missile. 

-  An  SM-6  BLK  1  FOT&E  mission  (D1H)  successfully 
engaged  a  target  that  was  using  electronic  attack  against 
the  Aegis  shipboard  radar  supporting  the  SM-6. 

-  An  SM-6  BLK  1  FOT&E  mission  (DIG)  failed  due  to  a 
failure-to-launch  (dud/misfire). 

NIFC-CA  FTS  Increment  I 

•  In  June  2015,  a  SM-6  BLK  I,  in  support  of  NIFC-CA  FTS 
testing,  successfully  engaged  a  full-scale  fighter  target  at 
White  Sands  Missile  Range,  New  Mexico. 

SM-6  BLK  IA 

•  In  November  2014,  the  Navy  successfully  conducted  a  land 
based  test  launch  of  the  pre-production  SM-6  BLK  IA  at  White 
Sands  Missile  Range,  New  Mexico.  The  missile  successfully 
engaged  a  subsonic  cruise  missile  target  overland.  This  was 
the  second  flight  test  of  the  SM-6  BLK  IA  configuration.  The 
Navy  plans  to  conduct  a  final  GTV  mission  in  FY16  using  the 
production  configuration  SM-6  BLK  IA. 

SM-6  Dual  I 

•  In  July  2015,  at  the  Pacific  Missile  Range  Facility,  Kauai, 
Hawaii,  the  MDA  and  Navy  successfully  conducted: 

-  A  Multi-Mission  Warfare  (MMW)  mission  1 .  In  this 
mission,  an  SM-6  missile  successfully  engaged  a 
short-range  ballistic  missile  target. 

-  An  SM-6  Dual  1  Air  Warfare  capability  retention  MMW 
mission  3.  In  this  mission,  an  SM-6  missile  successfully 
engaged  a  supersonic  high-diver  target. 

-  An  SM-6  Dual  1  Air  Warfare  capability  retention  MMW 
mission  4.  In  this  mission,  an  SM-6  missile  successfully 
engaged  a  subsonic  low-altitude  cruise  missile  target. 

•  The  Navy  conducted  these  tests  in  accordance  with  the 
DOT&E-approved  MDA  Integrated  Master  Test  Plan. 

Assessment 

•  During  FY15  flight  tests,  there  were  no  occurrences  of 
the  uplink/downlink  antenna  shroud  reliability  deficiency. 
DOT&E  and  the  Navy  will  continue  to  collect  data  on  this 
deficiency  throughout  FOT&E  flight-testing.  In  addition. 


there  were  no  observations  of  additional  anomalies  during 
these  tests. 

•  The  March  2015  SM-6  BLK  1  mission  DIG  misfire 
remains  under  investigation  by  the  Navy  with  no  root  cause 
determination  to  date. 

•  In  the  May  2013  SM-6  IOT&E  report,  DOT&E  assessed  SM-6 
BLK  I  as  suitable.  This  assessment  considered  combined 
data  from  the  IOT&E  and  developmental/operational  flight 
tests.  During  FY15  testing,  DOT&E  collected  additional 
reliability  data  and  assessed  the  SM-6  BLK  I  continues  to 
remain  suitable.  DOT&E  will  continue  to  collect  suitability 
and  effectiveness  data  throughout  SM-6  BLK  I  FOT&E 
testing  in  FY16,  as  well  as  during  all  SM-6  flight  testing  in 
support  of  NIFC-CA  FTS,  MDA,  and  Aegis  software  baseline 
development. 

•  The  performance  deficiency  discovered  during  IOT&E  and 
outlined  in  the  classified  IOT&E  report  remains  unresolved 
and  continues  to  affect  DOT&E ’s  final  assessment  of 
effectiveness.  The  Navy  is  assessing  several  options  for  a 
solution,  each  with  varying  degrees  of  complexity.  A  primary 
concern  is  to  ensure  the  solution  causes  no  degradation  to  the 
existing  SM-6  performance  envelope.  The  corrective  actions 
will  be  incorporated  into  production  of  the  SM-6  BLK  I  and 
BLK  IA  configurations  and  tested  during  FOT&E. 

•  In  FY16,  the  Navy  expects  to  demonstrate  the  maximum  range 
KPP  during  SM-6  FOT&E  and  Aegis  Baseline  9  operational 
testing  as  well  as  the  launch  availability  KPP. 

•  Upon  completion  of  the  current  phase  of  SM-6  FOT&E, 
the  Navy  will  have  conducted  sufficient  testing  to  allow  an 
assessment  of  the  SM-6  Capability  Production  Document 
performance  requirement  for  interoperability. 

•  NIFC-CA  FTS  Increment  1  test  events  conducted  during  FY15 
continue  to  be  basic  developmental  tests  not  conducted  in  an 
operationally  realistic  manner.  The  Navy  plans  to  continue 
testing  the  Increment  1  configuration  with  increasingly 
challenging  scenarios;  however,  no  operational  test  concept  or 
test  plans  for  NIFC-CA  FTS  increments  have  been  provided  to 
DOT&E. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  is  addressing 
the  previous  recommendations  from  FY14  to  complete 
corrective  actions  of  the  classified  performance  deficiency 
discovered  during  IOT&E  and  develop  a  flight  test  program 

to  test  those  corrective  actions;  however,  no  final  solution  has 
been  determined . 

•  FY15  Recommendation. 

1.  The  Navy  should  provide  DOT&E  with  an  operational 
test  concept  and  operational  test  plan  for  NIFC-CA  FTS 
Increment  2. 
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Surface  Electronic  Warfare  Improvement  Program 

(SEWIP)  Block  2 


Executive  Summary 

•  In  December  2015,  DOT&E  submitted  an  Early  Fielding 
Report  to  Congress  on  the  results  of  the  first  phase  of  the 
IOT&E  for  the  AN/SLQ-32  Electronic  Warfare  System 
(EWS)  equipped  with  the  Surface  Electronic  Warfare 
Improvement  Program  (SEWIP)  Block  2  upgrade.  The  Navy 
conducted  a  portion  of  the  IOT&E  from  August  22  through 
November  7,  2014,  on  USS  Bainbridge  (DDG  96)  in  the 
Virginia  Capes  operating  area. 

•  Analysis  of  the  available  IOT&E  data  showed  that,  while  the 
AN/SLQ-32  EWS  equipped  with  the  SEWIP  Block  2  upgrade 
provides  more  capability  in  detecting  and  classifying  threat 
emitters  than  the  legacy  AN/SLQ-32  EWS,  the  system  has 
problems  in  creating  multiple  tracks  from  a  single  emitter 
source,  in  addition  to  incorrectly  categorizing  emitter  tracks 
and  an  inability  to  hold  them  after  initial  detection.  These 
deficiencies  overwhelm  and  desensitize  the  operator  to 
potential  threats.  Other  deficiencies  include  numerous 
software  reliability  problems  and  a  lack  of  documentation 

in  addition  to  inadequate  crew  training  and  proficiency 
causing  the  Commander,  Operational  Test  and  Evaluation 
Force  (COTF)  test  team  to  provide  operationally  unrealistic 
assistance  to  the  crew,  which  unduly  influenced  some  test 
results.  Until  these  deficiencies  are  corrected,  the  AN/SLQ-32 
EWS  equipped  with  the  SEWIP  Block  2  upgrade  will  not  have 
operational  utility. 

•  The  Navy  is  correcting  these  deficiencies  and  is  scheduling 
another  phase  of  IOT&E  for  mid-2016. 

System 

•  SEWIP  is  an  incremental  development  program  that  is 
intended  to  improve  the  electronic  warfare  capability  on  all 
Navy  surface  combatants. 

•  The  SEWIP  Block  1  upgrade  focused  on  replacing  obsolete 
parts  in  the  AN/SLQ-32  and  incorporating  a  new  operator 


console,  a  specific  emitter  identification  capability,  and  an 
embedded  trainer. 

•  The  SEWIP  Block  2  upgrade  incorporates  a  new  antenna 
system  and  enhanced  processing  capabilities,  which  are 
intended  to  improve  the  AN/SLQ-32’s  passive  electronic 
support  capabilities. 

•  The  SEWIP  Block  3  upgrade,  which  is  in  early  development, 
will  incorporate  improvements  to  the  AN/SLQ-32  active 
electronic  attack  to  improve  ships  self-defense  capabilities. 

Mission 

Navy  surface  ship  crews  will  use  SEWIP  to  enhance  the  AN/SLQ 

32  EWS  anti-ship  missile  defense,  counter  targeting,  and 

counter-surveillance  capabilities  and  to  improve  the  system’s 

ability  to  collect  electronic  data. 

Major  Contractor 

Lockheed-Martin  -  Syracuse,  New  York 


Activity 

•  COTF  conducted  the  first  phase  of  IOT&E  from  August  22 
through  November  7,  2014,  on  USS  Bainbridge  (DDG  96)  in 
the  Virginia  Capes  operating  area. 

•  In  December  2015,  DOT&E  submitted  a  classified  Early 
Fielding  Report  to  Congress  on  the  results  of  the  first  phase 
of  IOT&E  of  the  AN/SLQ-32  EWS  equipped  with  the  SEWIP 
Block  2  upgrade. 

•  The  severity  of  the  identified  deficiencies  during  this  first 
phase  caused  the  Navy  to  schedule  another  phase  of  IOT&E 
for  mid-2016. 


Assessment 

•  Analysis  of  the  available  IOT&E  data  showed  that,  while 
the  AN/SLQ-32  EWS  equipped  with  the  SEWIP  Block 
2  upgrade  provides  more  capability  in  detecting  and 
classifying  threat  emitters  than  the  legacy  AN/SLQ-32 
EWS,  the  system  generates  multiple  tracks  from  a  single 
emitter  source  in  addition  to  incorrectly  categorizing  emitter 
tracks  and  an  inability  to  hold  them  after  initial  detection. 
These  deficiencies  overwhelm  and  desensitize  the  operator 
to  potential  threats.  Other  deficiencies  included  a  lack  of 
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documentation  and  numerous  software  reliability  problems 
that  included  display  freezes,  system  crashes,  and  unscheduled 
warm  and  cold  starts.  Until  these  deficiencies  are  corrected, 
the  AN/SLQ-32  EWS  equipped  with  the  SEWIP  Block  2 
upgrade  will  not  have  operational  utility 

•  Inadequate  crew  training  and  proficiency  caused  the  COTF 
test  team  to  provide  operationally  unrealistic  assistance  to 
the  crew,  which  unduly  influenced  some  test  results.  These 
included  providing  assistance  in  setting  up  displays  to  make 
threat  detection  easier  to  recognize,  showing  operators  which 
threats  needed  identification,  helping  operators  identify 
extraneous  emitter  contacts,  calibrating  the  system  when  the 
operators  failed  to  perform  the  calibration  the  system  required, 
and  resetting  the  system  in  the  form  of  unscheduled  warm  and 
cold  starts. 

•  The  Navy  plans  to  correct  these  deficiencies  and  demonstrate 
the  fixes  in  another  phase  oflOT&E  in  mid-2016. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  not 
resolved  the  following  SEWIP  Block  1  FY06  and  FY08 
previous  recommendations  to: 

1.  Review  and  modify  the  SEWIP  detection  and  classification 
algorithms  to  correct  deficiencies  discovered  while 


operating  in  dense  pulse  and  emitter  environments.  Verify 
the  correction  of  these  deficiencies  during  future  SEWIP 
OT&E. 

2.  Continue  to  collect  in-service  SEWIP  hardware  reliability 
data  to  gain  a  higher  degree  of  confidence  regarding 
achievement  of  this  requirement. 

3.  Continue  to  review  and  modify  the  SEWIP  software  to 
improve  its  reliability. 

4.  Develop  threat  representative  aerial  target/threat  seeker 
combinations  and/or  procure  actual  threat  anti-ship  cruise 
missiles  for  more  realistic  testing  of  future  SEWIP  block 
upgrades  and  other  EWSs. 

•  FY15  Recommendation. 

1.  The  Navy  should  conduct  the  next  IOT&E  phase  of  the 
AN/SLQ-32  EWS  equipped  with  the  SEWIP  Block  2 
upgrade  once  the  deficiencies  identified  in  the  first  IOT&E 
phase  are  corrected. 
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Surface  Ship  Torpedo  Defense  (SSTD) 
System:  Torpedo  Warning  System  (TWS)  and 
Countermeasure  Anti-Torpedo  (CAT) 


Executive  Summary 

•  DOT&E  submitted  a  classified  Early  Fielding  Report 
in  March  2015  on  the  Torpedo  Warning  System  (TWS) 
and  Countermeasure  Anti-Torpedo  (CAT)  system 
fielded  aboard  USS  Theodore  Roosevelt  (CVN  71). 

The  completed  testing  of  the  TWS  (passive)  and  CAT 
Engineering  Development  Model-2  (EDM-2),  powered 
by  Stored  Chemical  Energy  Propulsion  System  (SCEPS), 
fielded  aboard  Theodore  Roosevelt  (and  previously  on 
USS  George  H.  W.  Bush  (CVN  77)  has  not  demonstrated  an 
effective  capability  against  realistic  threat  country  torpedo 
attack  scenarios. 

•  The  Navy’s  Quick  Reaction  Assessment  (QRA)  of  TWS  and 
CAT  demonstrated  a  limited  capability  to  detect  and  home 
on  certain  types  of  single  torpedo  threats.  However,  these 
versions  of  the  system  have  not  been  fully  tested  and  the  Navy 
conducted  most  TWS  and  CAT  testing  in  areas  with  benign 
acoustic  conditions  when  compared  to  the  expected  threat 
operating  areas,  which  may  have  biased  the  results  high. 

Very  few  of  the  threat  surrogates  used  during  testing  were 
conducting  operationally  realistic  threat  torpedo  profiles  due 
to  safety  constraints.  Moreover,  the  lack  of  adequate  testing 
has  stymied  the  development  of  optimal  tactics,  techniques, 
and  procedures  and  assessment  of  system  availability  and 
reliability,  which  negatively  affects  the  system’s  suitability. 

•  The  Navy  installed  a  third  version  of  the  TWS  and 
the  CAT  EDM-2  powered  with  SCEPS  aboard  the 
USS  Harry  S.  Truman  (CVN  75)  during  FY15.  This 
version  is  similar  to  the  permanent  system  installed  on  the 
George  H.  W.  Bush  in  FY13  and  includes  the  TWS  Target 
Acquisition  Group,  the  Tactical  Control  Group  hardware, 
and  two  of  the  four  planned  CAT  Ready  Stow  Group  cradles. 
The  TWS  and  CAT  operational  software  is  the  same  version 
installed  for  Theodore  Roosevelt  deployment  to  the  Fifth  Fleet 
in  March  2015.  The  Navy  fielded  this  version  of  TWS  and 
CAT  when  the  Harry  S.  Truman  deployed  in  early  FY16. 

•  In  FY14  and  FY15,  the  Navy  installed  a  temporary-installation 
version  of  TWS  and  CAT  (designated  as  a  roll-on/roll-off 
system)  aboard  Theodore  Roosevelt  and  conducted  a  second 
QRA  in  the  Virginia  Capes  Operating  Areas.  Only  two  of 

the  four  planned  QRA  events  and  one  of  the  five  planned 
contractor  test  events  were  accomplished  due  to  several  factors 
including  poor  weather. 

•  The  TWS  program  of  record  includes  a  Towed  Active 
Acoustic  Source  (TAAS)  to  detect  torpedoes  using  active 
sonar,  but  after  exhibiting  multiple  critical  reliability  failures 


CAT  -  Countermeasure  Anti-Torpedo 
RSG  -  Ready  Stow  Group 
TAG  -  Target  Acquisition  Group 
TCG  -  Tactical  Control  Group 
TWS  -  Torpedo  Warning  System 

during  the  FY15  QRA,  the  Navy  was  unable  to  field  the 
TAAS  aboard  Theodore  Roosevelt  as  planned,  and  so,  like 
George  H.  W.  Bush,  the  ship  deployed  with  a  passive-only 
TWS. 

•  The  Navy  is  currently  installing  a  roll-on/roll-off  version  of 
TWS  and  CAT  on  USS  Dwight  D.  Eisenhower  (CVN  69) 
to  support  her  next  deployment  and  is  in  various  stages  of 
planning  and  installing  the  permanent  version  of  the  TWS  and 
CAT  early  fielded  hardware  on  selected  CVNs  before  their 
next  deployments. 

System 

•  Surface  Ship  Torpedo  Defense  is  a  system-of-systems  that 
includes  two  new  sub-programs:  the  TWS  (an  Acquisition 
Category  III  program)  and  CAT  (will  not  become  an 
acquisition  program  until  FY16).  Combined,  TWS  and  CAT 
are  referred  to  as  the  Anti-Torpedo  Torpedo  (ATT)  defense 
system. 

•  TWS  is  being  built  as  an  early  warning  system  to  detect, 
localize,  classify,  and  alert  on  incoming  threat  torpedoes  and 
consists  of  three  major  subsystems: 

-  The  Target  Acquisition  Group  consists  of  a  towed 
acoustic  array,  tow  cable,  winch,  power  supply,  and  signal 
processing  equipment.  Data  from  the  array  and  the  ship’s 
radar  system  are  processed  into  contact  tracks  and  alerts  to 
be  forwarded  to  the  Tactical  Control  Group.  The  array  will 
be  capable  of  both  passive  and  active  sonar  operations. 

-  The  Tactical  Control  Group  consists  of  duplicate  consoles 
on  the  bridge  and  Combat  Direction  Center  (on  CVNs) 
that  displays  contacts,  issues  torpedo  alerts  to  the  crew, 
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and  automatically  develops  CAT  placement  presets  using 
information  sent  from  the  Target  Acquisition  Group. 

The  operator  will  use  this  console  to  manage  the  threat 
engagement  sequence  and  command  CAT  launches. 

-  The  Ready  Stow  Group  will  consist  of  the  steel  cradles 
housing  the  CATs.  The  permanent  system  consists  of 
four  steel  cradles  and  associated  electronics,  each  housing 
six  ATTs  at  different  locations  (port/starboard  and  fore  and 
aft  on  the  CVN). 

•  CAT  is  a  hard-kill  countermeasure  intended  to  neutralize  threat 
torpedoes  and  consists  of  the  following: 

-  The  ATT  is  a  6.75-inch  diameter  interceptor  designed 
for  high-speed  and  maneuverability  to  support  rapid 
engagement  of  the  threat  torpedo. 

-  The  All-Up  Round  Equipment  consists  of  a  nose  sabot,  ram 
plate,  launch  tube,  muzzle  cover,  breech  mechanism,  and 
energetics  to  encapsulate  and  launch  the  ATT. 

-  The  tactical  CAT  is  powered  by  a  SCEPS.  The 
battery-powered  electric  motor  CAT  is  for  test  purposes 
only.  EDM-2  is  the  current  hardware  version  of  the  CAT. 

•  The  Navy  developed  a  temporary  version  of  TWS  and  CAT 
(designated  a  roll-on/roll-off  system)  in  addition  to  the 
permanent-installation  version.  The  Navy  intends  for  this 
version  to  provide  the  same  function  as  the  permanent  one. 

-  The  Ready  Stow  Group  steel  cradles  are  replaced  by  two 
lighter-weight  and  less-robust  aluminum  Launch  Frame 
Assemblies  that  each  hold  four  CATs. 


-  The  Tactical  Control  Group  consists  of  two  consoles 
contained  in  a  container  express  box  located  on  the 
carrier’s  hangar  deck. 

-  The  towed  acoustic  array,  tow  cable,  and  winch  are 
permanently  installed  on  the  carrier’s  fantail.  The  other 
components  of  the  Target  Acquisition  Group  are  contained 
in  the  container  express  box  located  on  the  hangar  deck. 

Mission 

Commanders  of  nuclear-powered  aircraft  carriers  and  Combat 
Logistic  Force  ships  will  use  the  Surface  Ship  Torpedo  Defense 
system  to  defend  against  incoming  threat  torpedoes. 

Major  Contractors 

TWS 

•  Ultra  Electronics-3Phoenix  -  Chantilly,  Virginia 

•  Alion  Science  and  Technology  -  New  London,  Connecticut 

•  In-Depth  Engineering  -  Fairfax,  Virginia 

CAT 

•  Pennsylvania  State  University  Applied  Research 
Laboratory  -  State  College,  Pennsylvania 

•  Pacific  Engineering  Inc.  (PEI)  -  Lincoln,  Nebraska 


Activity 

•  The  Navy  installed  a  third  version  of  the  TWS  and  the  CAT 
EDM-2  powered  with  SCEPS  aboard  Harry  S.  Truman  during 
FY15.  This  version  is  similar  to  the  FY13  installation  aboard 
George  H.  W.  Bush  and  includes  the  permanent  installation  of 
the  TWS  Target  Acquisition  Group  and  the  Tactical  Control 
Group  hardware  and  two  of  the  four  planned  CAT  Ready  Stow 
Group  steel  cradles.  The  TWS  and  CAT  operational  software 
is  the  same  version  as  installed  aboard  Theodore  Roosevelt  s 
roll-on/roll-off  system  in  FY 14.  The  Navy  fielded  this  version 
of  TWS  and  CAT  when  the  USS  Harry  S.  Truman  deployed  in 
early  FY16.  The  Navy  is  currently  installing  a  roll-on/roll-off 
version  aboard  Dwight  D.  Eisenhower  to  support  her  next 
deployment  and  is  in  various  stages  of  planning  and  installing 
the  permanent  version  of  the  TWS  and  CAT  early  fielded 
hardware  on  selected  CVNs  before  their  next  deployment. 

•  The  Navy  installed  an  initial  version  of  TWS  and  the  CAT 
EDM-2  aboard  George  H.  W.  Bush  and  conducted  a  QRA  in 
the  Virginia  Capes  Operating  Areas  in  November  2013.  The 
George  H.  W.  Bush,  which  deployed  with  the  TWS  system 
and  the  CAT  EDM-2  with  the  SCEPS  propulsion  system, 
returned  from  deployment  to  Fifth  Fleet  Operating  Areas  in 
November  2014. 

•  DOT&E  submitted  a  classified  Early  Fielding  Report  in 
April  2014  on  the  TWS  and  CAT  system  fielded  aboard 


George  H.  W.  Bush.  DOT&E  submitted  a  classified  update 
to  the  Early  Fielding  Report  in  August  2014,  following  the 
Navy’s  discovery  of  an  anomaly  in  the  CAT’s  Safety  and 
Arming  device.  The  Navy  developed  a  correction  for  the 
anomaly  in  the  CAT  Safety  and  Arming  device  but  could 
not  install  the  correction  in  the  fielded  CATs  due  to  safety 
concerns  and  USS  George  H.  W.  Bush  s  operational  schedule. 

•  In  FY14  and  FY15,  the  Navy  installed  a  roll-on/roll-off 
version  of  TWS  and  CAT  aboard  Theodore  Roosevelt  and 
conducted  a  second  QRA  in  the  Virginia  Capes  Operating 
Areas  in  November  2014.  The  QRA  event  was  conducted  in 
conjunction  with  a  contractor  test  event.  Only  two  of  the  four 
planned  QRA  events  and  one  of  the  five  planned  contractor 
test  events  were  accomplished  due  to  several  factors  including 
poor  weather.  During  each  completed  event,  a  single  surrogate 
threat  torpedo  was  fired  at  Theodore  Roosevelt  for  the  TWS 
system  to  detect  and  target.  Theodore  Roosevelt  s  crew,  with 
the  contractor  support  that  accompanied  the  ship  on  their 
deployment,  engaged  the  threat  torpedo  surrogate  with  a  CAT 
during  some  of  the  events.  All  CATs  that  were  fired  used 
electric  propulsion. 

•  The  Navy’s  contractor  redesigned  the  active  source  and  the 
Navy  planned  to  field  it  aboard  Theodore  Roosevelt,  but  there 
was  insufficient  time  and  resources  to  complete  development 
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and  testing  prior  to  deployment.  The  redesigned  TAAS  failed 
during  the  November  2014  QRA,  exhibiting  new  failure 
modes  and  failing  in  a  similar  fashion  to  the  older  one. 

•  The  Navy  fielded  the  roll-on/roll-off  version  of  the 
TWS  and  the  CAT  EDM-2  powered  by  SCEPS  aboard 
Theodore  Roosevelt  when  she  deployed  in  March  2015 
to  Fifth  Fleet  Operating  Areas.  The  ship  deployed  with  a 
passive-only  TWS  since  the  Navy  was  unable  to  deliver  a 
functioning  TAAS.  DOT&E  submitted  a  classified  Early 
Fielding  Report  on  the  roll-on/roll-off  version  of  TWS  and 
CAT  in  March  2015. 

•  Theodore  Roosevelt  returned  from  deployment  in 
November  2015.  Due  to  operational  limitations,  the  crew 
rarely  deployed  the  TWS  array  during  the  deployment  or  while 
in  the  Fifth  Fleet  Operating  Areas. 

•  In  February  and  April  2015,  the  Navy  and  Pennsylvania  State 
University  Applied  Research  Laboratory  conducted  contractor 
testing  of  CAT  on  the  Dabob  Bay,  Washington,  and  the 
Nanoose  Bay,  British  Columbia,  Canada,  acoustic  tracking 
ranges.  The  testing  included  runs  to  develop  CAT’s  salvo 
capability  and  to  evaluate  the  CAT’s  long-range  passive  fuzing 
capability. 

•  In  August  2015,  the  Navy  and  the  Naval  Surface  Warfare 
Center  Carderock  conducted  warhead  and  Safety  and  Arming 
device  airburst  testing  at  Fort  A.P  Hill,  Virginia.  This  testing 
verified  the  arming,  fuzing,  and  firing  of  the  ATT  warhead. 

•  In  August  2015,  the  Navy  and  Pennsylvania  State  University 
Applied  Research  Laboratory  conducted  CAT  contractor 
testing  on  the  Dabob  Bay,  Washington,  acoustic  range.  The 
testing  supported  developing  the  CAT’s  salvo  capability. 

•  In  August  and  September  2015,  the  Navy,  Pennsylvania 
State  University  Applied  Research  Laboratory,  and  3Phoenix 
conducted  contractor  testing  of  the  TWS  and  CAT  on  the 
Nanoose  Bay,  British  Columbia,  Canada,  acoustic  tracking 
range.  To  accomplish  this  testing,  the  Navy  removed  the  TWS 
array  from  George  H.  W.  Bush  and  installed  the  repaired  TAAS. 
This  testing  supported  developing  the  TWS  TAAS  active 
waveforms  and  developing  the  end-to-end  TWS  detection  and 
targeting  and  CAT  intercept  capability  against  threat  torpedo 
salvos.  On  the  third  day,  the  TAAS  array  developed  a  short  on 
one  of  the  two  TAAS  transducer  strings.  The  TWS  contractor 
isolated  the  grounded  string  of  transducers  and  continued  the 
test  with  the  remaining  half  of  the  TAAS  transducers  and  the 
passive  hydrophones. 

•  The  Navy  deployed  the  TWS  (passive  only  system)  and 
CAT  EDM-2  powered  by  SCEPS  aboard  Harry  S.  Truman 
when  she  deployed  in  1QFY16.  Like  the  previous 
deployments,  3Phoenix  contractor  personnel  deployed  with  the 
Harry  S.  Truman  to  operate  the  TWS  system  and  to  train  the 
crew. 

•  During  FY15,  the  Navy  and  DOT&E  continued  development 
of  an  enterprise  Test  and  Evaluation  Master  Plan  (TEMP) 
for  the  TWS  and  CAT  systems.  The  Navy  made  their  TWS 


Milestone  B  decision  without  a  TEMP  and  is  not  planning  to 
make  the  CAT  system  an  acquisition  program  until  FY16. 

Assessment 

•  The  completed  testing  of  TWS  (passive)  and  CAT  EDM-2, 
powered  by  SCEPS,  fielded  aboard  George  H.  W.  Bush 
and  Theodore  Roosevelt ,  has  not  demonstrated  an  effective 
capability  against  realistic  threat  torpedo  attack  scenarios. 

The  Navy’s  QRAs  of  TWS  and  CAT  demonstrated  limited 
capability  to  detect  and  home  on  certain  types  of  single 
torpedo  threats.  However,  these  versions  of  the  system  have 
not  been  fully  tested  and  the  Navy  conducted  most  TWS  and 
CAT  testing  in  areas  with  benign  acoustic  conditions  when 
compared  to  the  expected  threat  operating  areas,  which  may 
have  biased  the  results  high.  Very  few  of  the  threat  surrogates 
used  during  testing  were  conducting  operationally  realistic 
threat  torpedo  profiles  due  to  safety  constraints. 

•  Moreover,  the  lack  of  adequate  contractor  and  developmental 
testing  has  stymied  the  development  of  system  detection; 
tracking  and  alerting  software;  operator  tactics,  techniques, 
and  procedures;  and  assessments  of  system  availability 

and  reliability.  Although  the  Navy  and  Pennsylvania  State 
University  Applied  Research  Laboratory  are  able  to  conduct 
independent  structured  CAT  testing,  3Phoenix’s  TWS  testing 
is  limited  because  the  prototype  TWS  arrays  are  rapidly  fielded 
to  the  deploying  CVN,  leaving  the  3Phoenix  contractors 
without  a  full  system  to  continue  development.  Further,  the 
CVN’s  assigned  operational  areas  in  the  Fifth  Fleet  Theater 
and  the  deployed  operational  profile  limited  use  of  the  array. 
Thus,  the  amount  of  TWS  data  collected  during  the  CVN 
deployments  is  less  than  expected. 

•  The  Navy  recently  delayed  the  Initial  Operational  Capability 
of  the  TWS  and  CAT  from  2018  to  2022.  The  Navy  required 
the  Program  Office  to  deliver  an  early  capability  for  the  early 
fielded  TWS  and  CAT.  DOT&E  assesses  the  Navy  delivered 
a  very  limited  capability,  but  that  the  early  fielding  approach 
is  causing  development  delays  and  has  resulted  in  a  3-4  year 
delay  in  delivering  the  Capability  Development  Document 
required  torpedo  defense  capability  to  the  CVNs.  Because 
contractor  resources  are  limited,  the  same  Navy  and  contractor 
personnel  are  building  TWS  and  CAT  hardware  sets  and 
providing  installation  and  in-service  support  to  the  CVNs, 
while  also  attempting  to  develop  the  required  TWS  and  CAT 
capability. 

•  The  Navy’s  decision  to  add  a  highly-trained  contractor  as 
the  acoustic  operator  to  supplement  the  automated  detection 
and  alerting  functions  of  TWS,  improved  threat  detection 
performance  during  the  FY14  and  FY15  QRAs.  DOT&E 
assesses  the  majority  of  the  TWS’s  detection  and  alerting 
capability  is  a  result  of  the  contractor  acoustic  operators 
monitoring  the  TWS  displays  to  provide  early  alerts  on  threat 
torpedoes.  However,  the  test  areas  did  not  offer  the  same 
number  of  opportunities  for  false  alerts  as  expected  in  the 
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threat  area;  thus,  it  is  not  known  if  the  presence  of  the  operator 
could  also  reduce  the  false  alert  rate.  For  safety  reasons,  the 
QRA  testing  was  highly  structured  and  allowed  the  operators 
to  focus  on  torpedo  detections  and  firing  the  CAT.  Therefore, 
QRA  testing  was  inadequate  to  resolve  the  rate  of  false  alerts 
or  their  effect  on  mission  accomplishment. 

•  George  H.  W.  Bush  ,’v  and  Theodore  Roosevelt  s  deployments 
were  useful  in  identifying  TWS  false  alert  sources,  but 
system  development  done  using  these  data  needs  to  be 
assessed  in  testing  to  include  the  presence  of  both  threat 
torpedo  surrogates  and  assets  that  may  cause  false  alerts 
simultaneously. 

•  During  contractor  testing  and  the  FY14  and  FY15  QRAs,  a 
properly  targeted  CAT  EDM-2  demonstrated  a  capability  to 
detect  and  home  on  a  single  surrogate  torpedo.  Flowever, 
nearly  all  of  the  surrogate  threat  torpedoes  and  CATs  were 
operating  deeper  than  most  expected  threat  torpedoes  due  to 
safety  reasons. 

•  During  FY15,  the  Navy  and  Pennsylvania  State  University 
Applied  Research  Laboratory  conducted  shallower  torpedo 
salvo  scenarios  that  allowed  the  CATs  to  track  and  attack  the 
surrogate  threat  torpedoes  in  more  challenging  areas  of  the 
water  column.  The  Navy  designed  these  scenarios  to  develop 
salvo  tactics  where  the  CATs  conduct  a  coordinated  attack 
on  a  threat  torpedo  salvo.  During  other  events,  the  Navy, 
Pennsylvania  State  University  Applied  Research  Laboratory, 
and  3 Phoenix  conducted  scripted  test  events  to  create  TWS 
and  CAT  detection  opportunities  and  obtain  data  for  TWS 
waveforms  and  detection,  tracking  and  targeting  software 
development,  and  CAT  salvo  development.  Analysis  of  the 
test  data  is  in  progress;  however,  observations  showed  a  mix 
of  intercept  and  miss  results  in  easier  environmental  areas. 
These  tests  were  conducted  with  TWS  and  CAT  developmental 
software  that  will  not  be  fielded.  The  Navy  conducted  these 
tests  at  Dabob  Bay,  Washington,  and  Nanoose  Bay,  British 
Columbia,  Canada.  The  Navy  has  not  collected  adequate  data 
to  assess  CAT’s  overall  ability  to  neutralize  these  threat  salvos. 

•  During  the  FY15  Nanoose  Bay  testing,  the  TWS  TAAS 
operated  reliably  and  demonstrated  a  capability  to  alert  on  and 
to  track  targets  during  the  first  two  days  of  the  test,  however 
half  of  the  array’s  transducers  grounded  on  the  third  day  of 
testing.  The  contractor  isolated  one  of  the  TAAS  transducer 
strings,  which  allowed  testing  to  continue.  Analysis  of  the 
data  is  in  progress,  but  initial  observations  suggest  that  the 
array  was  still  able  to  provide  active  detections  when  half  of 


the  TAAS  transducers  were  functioning,  albeit  with  a  degraded 
accuracy. 

•  The  Navy  developed  and  fielded  a  correction  for  the  anomaly 
in  the  CAT  Safety  and  Arming  device.  Analysis  is  still 

in  progress,  but  observations  from  the  airburst  testing  of 
the  Safety  and  Arming  Device  and  warhead  indicates  the 
correction  was  effective. 

•  Testing  completed  in  FY15  indicates  the  new  TAAS  has  both 
hardware  and  software  reliability  deficiencies,  which  the 
Navy’s  contractors  are  investigating. 

•  Additional  information  concerning  the  testing  of  TWS  and 
CAT  performance  aboard  Theodore  Roosevelt  is  included  in 
DOT&E’s  March  2015  classified  Early  Fielding  Report. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  has  made 
some  progress  on  the  FY13  and  FY14  recommendations. 
Flowever,  the  Navy  should  still: 

1.  Complete  the  TEMP  for  the  TWS  and  CAT  system  and  an 
LFT&E  strategy  for  the  ATT  lethality  as  soon  as  possible. 

2.  Conduct  additional  testing  in  challenging,  threat 
representative  environments. 

3.  Conduct  additional  CAT  testing  using  operationally 
realistic  threat  target  profiles  closer  to  the  surface  to  assess 
the  CAT’s  tenninal  homing,  attack,  and  fuzing  within  the 
lethality  range  of  the  warhead. 

4.  Investigate  test  methods  designed  to  reduce  or  eliminate 
the  safety  limitations  that  have  previously  prevented 
testing  against  operationally  realistic  target  scenarios.  The 
Navy  should  consider  using  geographic  separation,  range 
boundaries,  and  shallow  draft  ships  for  future  TWS  and 
CAT  testing. 

5.  Continue  to  investigate,  correct,  and  retest  deficiencies 
identified  with  the  active  source  before  planning  to  field 
TAAS. 

•  FY 15  Recommendations.  The  Navy  should: 

1.  Adequately  resource  the  TWS  program  to  build  dedicated 
test  assets  and  conduct  adequate  dedicated  contractor  and 
developmental  testing. 

2.  Adequately  resource  the  Program  Office  and  its  contractors 
to  conduct  TWS  and  CAT  system  development  and  testing. 

3.  Investigate  and  implement  the  14  outstanding 
recommendations  in  the  March  2015  DOT&E  Early 
Fielding  Report. 
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Surveillance  Towed  Array  Sensor  System  (SURTASS) 
and  Compact  Low  Frequency  Active  (CLFA)  Sonar 


Executive  Summary 

•  The  Commander,  Operational  Test  and  Evaluation  Force 
completed  IOT&E  on  Surveillance  Towed  Array  Sensor 
System  (SURTASS)  and  Compact  Low  Frequency  Active 
(CLFA)  Sonar  in  January  2016.  Testing  was  conducted  in 
accordance  with  a  DOT&E-approved  test  plan. 

•  DOT&E  submitted  an  IOT&E  report  for  SURTASS/CLFA 
in  January  2015.  It  concluded  that  SURTASS/CLFA  was  not 
operationally  effective  for  wartime  Anti-Submarine  Warfare 
(ASW)  and  that  SURTASS/CLFA  has  significant  cybersecurity 
vulnerabilities.  SURTASS/CLFA  was  operationally  suitable. 

System 

•  SURTASS/CLFA  is  a  low-frequency,  passive  and  active, 
acoustic  surveillance  system  installed  on  Victorious  class 
tactical  auxiliary  general  ocean  surveillance  (T-AGOS)  ships 
as  a  component  of  the  Integrated  Undersea  Surveillance 
System. 

•  SURTASS  provides  passive  detection  of  nuclear  and  diesel 
submarines  and  enables  real-time  reporting  of  surveillance 
information  to  ASW  commanders. 

•  CLFA  is  a  low-frequency,  active  sonar  system  developed  to 
provide  an  active  detection  capability  of  quiet  submarines 
operating  in  environments  that  support  long-range 
propagation. 

•  The  system  consists  of: 

-  A  T-AGOS  host  ship  with  array-handling  equipment 

-  A  towed  vertical  string  of  active  acoustic  projectors 

-  A  towed  horizontal  twin  line  (TL-29A)  passive  sonar  array 

-  An  Integrated  Common  Processor  for  processing  active 
and  passive  acoustic  data 

-  A  High-Frequency  Marine-Mammal  Monitoring  active 
sonar  used  to  ensure  local  water  space  is  free  of  marine 
mammals  prior  to  and  during  low-frequency  active 
transmission 

-  A  communications  segment  to  provide  connectivity  to 
shore-based  Integrated  Undersea  Surveillance  System 
processing  facilities  and  to  fleet  ASW  commanders 


Mission 

Maritime  Component  Commanders: 

•  Employ  Victorious  class  T-AGOS  ships  equipped  with 
SURTASS/CLFA  systems  to  provide  long-range  active 
and  passive  ASW  detection,  classification,  and  tracking  of 
submarines  in  support  of  Carrier  Strike  Group  and  theater 
ASW  operations. 

•  Use  SURTASS/CLFA  to  provide  blue  force  ASW  screening 
and  threat  submarine  localization  information  to  theater  ASW 
commanders  to  support  coordinated  prosecution  of  detected 
threat  submarines. 

Major  Contractors 

•  Integrated  Common  Processor:  Lockheed  Martin  -  Manassas, 
Virginia 

•  CLFA  Projectors:  BAE  -  Nashua,  New  Hampshire 

•  TL-29A  Towed  Arrays:  Lockheed  Martin  -  Syracuse,  New 
York 


Activity 

•  In  January  2015,  the  Commander,  Operational  Test  and 
Evaluation  Force  completed  IOT&E  for  SURTASS/CLFA. 
Testing  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan  and  included: 

-  ASW  area  search  operations  that  supported  coordinated 
theater  ASW  during  fleet  exercises  Valiant  Shield  12  and 
Valiant  Shield  14 


-  Dedicated,  at  sea  test  phases  in  2012  and  2014  that 
obtained  performance  data  necessary  to  characterize 
detection  capability  against  long-range  submarine 
approaches 

-  A  cybersecurity  assessment  in  January  2015 
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•  In  January  2016,  DOT&E  submitted  a  classified  IOT&E  report 
for  SURTASS/CLFA  based  on  observations  and  data  collected 
during  operational  testing. 

•  One  engineering  development  model  and  two  production 
CLFA  systems  were  available  for  operation  on  three  of  the 
four  Western  Pacific -based  Victorious  class  T-AGOS  ships 
during  FY15. 

Assessment 

•  The  DOT&E  classified  IOT&E  report  concluded  the 
following: 

-  Testing  was  adequate  to  assess  operational  effectiveness 
and  suitability. 

-  SURTASS/CLFA  was  not  operationally  effective  for 
wartime  ASW.  The  fleet  did  not  demonstrate  sufficient 
capability  to  correlate  non-submarine  CLFA  detections  that 
were  reported  as  possible  submarines  to  real-time  surface 
ship  positions.  Failure  to  exclude  surface  ship  detections, 
coupled  with  limited  numbers  of  available  ASW-capable 
assets,  will  not  support  fleet  prosecution  of  CLFA 
submarine  localizations.  Further  details  of  the  observed 
deficiencies  are  available  in  the  classified  report. 

-  Although  not  the  primary  focus  of  this  limited  operational 
testing,  SURTASS/CLFA  can  support  coordinated  ASW 
during  peacetime.  Long-range  active  detection  and 
localization  capability  exceeds  that  of  all  non-SURTASS 
platforms.  Without  the  risk  of  imminent  attack, 


SURTASS/CLFA  limitations  can  likely  be  overcome 
through  extended  analysis  and  comprehensive  integration 
of  other  ASW  platform  and  sensor  data. 

-  SURTASS/CLFA  is  operationally  suitable,  but  exhibits 
reliability  deficiencies  during  system  initialization. 
Subsequent  to  a  successful  start-up  of  both  the  passive  and 
active  sonar  arrays,  SURTASS/CLFA  demonstrated  that  it 
can  operate  without  failure  for  extended  and  operationally 
relevant  periods  of  time. 

-  Cybersecurity  evaluation  identified  significant  problems, 
which  are  classified. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Navy  should 
continue  to  address  the  FY13  recommendation  to  improve 
procedures  and  training  for  correlating  CLFA  non-submarine, 
active  detections  with  real-time  surface  vessel  positions. 

•  FY 15  Recommendations.  The  Navy  should: 

1 .  Mitigate  identified  cybersecurity  vulnerabilities  as  soon 
as  feasible  for  deployed  SURTASS/CLFA  and  incorporate 
long-term  corrections  within  future  increments  of 
SURTASS/CLFA. 

2.  Address  reliability  failure  modes  observed  during 
operational  testing. 

3.  Address  the  seven  classified  recommendations  listed  in  the 
January  2016  IOT&E  report. 
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Air  Force  Prog 


AC-130J  Ghostrider 


Executive  Summary 

•  U.S.  Special  Operations  Command  (USSOCOM)  is 
developing  AC-130J  through  the  integration  of  a  modular 
Precision  Strike  Package  (PSP)  onto  existing  MC-130J 
aircraft.  An  earlier  version  of  the  PSP  was  previously 
developed  and  tested  on  several  AC- DOW  aircraft  since  2009 
and  fielded  in  2010. 

•  An  in-flight  Class  A  mishap  in  April  2015,  which  was  caused 
by  a  second  departure  from  controlled  flight,  grounded  the 
only  available  aircraft  and  truncated  the  already-delayed 
developmental  test  and  evaluation  (DT&E)  and  concurrent 
operational  assessment  (OA). 

•  Aircraft  #2  was  delivered  in  the  Block  10  configuration  in 
June  2015. 

•  DT&E  and  the  OA  ended  in  July  2015  on  aircraft  #2  without 
completing  either  test  plan  due  to  both  delays  caused  by  the 
departures  from  controlled  flight  and  limitations  on  weapon 
employment  (PSP  integration  problems). 

•  Because  of  significant  technical  problems  discovered 
during  DT&E,  the  OA  indicated  the  Block  10  AC-130J 
was  at  risk  of  not  being  ready  for  IOT&E.  Per  DOT&E’s 
recommendation,  the  program  will  instead  conduct  IOT&E  in 
3QFY17  on  Block  20  aircraft,  which  are  a  more  operationally 
representative  configuration  of  the  aircraft  desired  for  Initial 
Operational  Capability. 

•  The  program  will  conduct  an  Operational  Utility  Evaluation 
(OUE)  on  the  Block  10  aircraft  to  support  an  early  fielding 
decision  and  a  Low-Rate  Initial  Production  decision  at 
Milestone  C  in  3QFY16. 

System 

•  The  AC-130J  is  a  medium-sized,  multi-engine,  tactical  aircraft 
with  a  variety  of  sensors  and  weapons  for  air-to-ground  attack. 

•  U S  SOCOM  is  developing  the  AC- 1 30 J  by  integrating  a 
modular  PSP  onto  existing  MC-D0J  aircraft,  and  replacing 
the  MC-D0J  refueling  pods  with  weapon  racks.  USSOCOM 
continues  to  develop  new  PSP  capabilities  on  legacy 

AC- DOW  aircraft  in  parallel  before  they  are  introduced  on  the 
AC-D0J  in  an  evolutionary  acquisition  approach: 

-  The  Block  10  AC-D0J  PSP  provides  a  weapons  suite  that 
includes  an  internal,  pallet  mounted  30  mm  side-firing 
chain  gun;  wing-mounted  GPS-guided  Small  Diameter 
Bombs;  and  Griffin  laser-guided  missiles  mounted 
internally  and  launched  through  the  rear  cargo  door. 

-  The  PSP  also  provides  two  electro  optical/  infrared 
sensor/laser  designator  pods  and  multiple  video,  data,  and 
communication  links. 

-  A  dual-console  Mission  Operator  Pallet  (MOP)  in  the 
cargo  bay  controls  all  PSP  subsystems  with  remote 
displays  and  control  panels  (including  master  arm  and 


consent  switches  and  a  gun  trigger)  on  the  flight  deck. 

An  interim,  limited-functionality,  carry-on  flight  deck 
workstation  for  a  Fire  Control  Officer  (FCO)  has  been 
added  to  the  Block  10  AC-130J. 

-  Block  20  AC-130J  will  add,  at  a  minimum,  a  105  mm  gun, 
a  pilot  helmet-mounted  tactical  display,  and  Large  Aircraft 
Infrared  Countermeasures.  The  aircrew  will  increase  from 
seven  to  nine.  The  first  Block  20  configuration  is  expected 
to  be  delivered  on  aircraft  #4  in  4QFY16. 

-  Future  updates  in  Blocks  30,  40,  and  50  are  expected  to 
include  a  permanent  Fire  Control  Officer  station,  additional 
radar  to  provide  all-weather  engagement  capability, 
wing-mounted  HELLFIRE  missiles,  laser-guided  Small 
Diameter  Bombs,  and  radio-frequency  countermeasures. 

•  The  Block  10  AC-130J  retains  all  survivability  enhancement 
features  found  on  the  HC/MC-130J  aircraft. 

-  Susceptibility  reduction  features  include  the  AN/ALR-56M 
radar  warning  receiver,  the  AN/AAR-47(V)2  missile 
warning  system,  and  the  AN/  ALE  47  countermeasure 
dispensing  system. 

-  Vulnerability  reduction  features  include  fuel  system 
protection  (fuel  tank  foam  to  protect  from  ullage 
explosion),  redundant  flight-critical  components,  and 
armor  to  protect  the  crew  and  the  oxygen  supply. 

•  The  AC-130J  will  replace  legacy  AC- 13 0H/U  aircraft. 

Mission 

The  Joint  Task  Force  or  Combatant  Commander  will  use  the 

AC-130J  to: 

•  Provide  persistent  and  precision  strike  operations  for  the 
missions  of  close  air  support  and  air  interdiction 

•  Provide  battlespace  wide  area  surveillance,  target  geo-location, 
and  precision  munition  application 
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Major  Contractor 

Lockheed  Martin  -  Bethesda,  Maryland 


Activity 

•  The  18th  Flight  Test  Squadron  conducted  an  OA  of  the 
Block  10  AC-130J  from  October  2014  to  July  2015.  Testing 
consisted  of  12  sorties  and  47  flight  hours  during  the  DT&E 
period. 

•  Aircraft  #1  experienced  a  Class  A  mishap  in  April  2015  during 
supplemental  flying  and  handling  qualities  testing  and  has 
been  grounded  since  the  incident. 

•  Lockheed  Martin  delivered  the  second  Block  10  AC-130J  to 
USSOCOM  in  June  2015.  Aircraft  #2  became  the  test  article 
for  the  remainder  of  DT&E  and  one  final  OA  flight. 

•  The  program  concluded  Block  10  DT&E  in  July  2015  after 
completing  97  flights  comprising  of  307  hours. 

•  Aircraft  #3  began  modification  in  August  2015  and  is  expected 
to  be  complete  in  the  Block  10  configuration  in  March  2016. 
Aircraft  #4  began  modification  in  September  2015  and  is 
expected  to  be  complete  in  the  Block  20  configuration  by 
October  2016. 

•  Air  Force  Special  Operations  Command  stood  up  Detachment 
2  of  the  1st  Special  Operations  Group  at  Hurlburt  Field, 
Florida,  as  the  first  unit  to  receive  the  AC-130J.  Two  crews 
from  Detachment  2  began  training  on  the  AC-130J  in 
August  2015  to  support  operational  testing  by  December  2015. 

•  The  Program  Office  submitted  an  updated  Test  and  Evaluation 
Master  Plan  to  DOT&E  to  support  a  Milestone  C  decision 

in  4QFY15  and  IOT&E  on  Block  10  starting  in  1QFY16. 

At  DOT&E’s  recommendation,  however,  an  OUE  will  be 
conducted  on  Block  10  instead  of  an  IOT&E  to  support  a 
USSOCOM  early  fielding  decision  for  the  Block  10  aircraft 
and  a  Low-Rate  Initial  Production  decision  at  Milestone  C. 

•  The  18th  Flight  Test  Squadron  will  conduct  IOT&E  on  a 
Block  20  aircraft  in  3QFY17.  Milestone  C  has  been  delayed 
to  at  least  3QFY16  in  order  to  consider  results  from  the 
Block  10  OUE  and  ongoing  deficiency  resolution  efforts. 

•  The  780th  Test  Squadron  is  working  with  DOT&E  to  develop 
a  weapons  lethality  test  plan  for  the  Griffin  missile  and  30  mm 
gun  against  representative  targets  to  support  the  AC-130J 
lethality  evaluation. 

•  The  U.S.  Air  Force  Combat  Effectiveness  and  Vulnerability 
Analysis  Branch  is  executing  the  Ballistic  Vulnerability 
Analysis,  Anti-Aircraft  Artillery  Susceptibility  Analysis, 
Proximity  Burst  Analysis,  and  Occupant  Casualty  Analysis, 
in  accordance  with  the  DOT &E-approved  LFT &E  strategy,  to 
support  the  AC-130J  survivability  evaluation. 

Assessment 

•  The  program  took  steps  to  address  system  integration 
problems  discussed  in  last  year’s  annual  report: 

-  The  mounts  for  the  electro-optical/infrared  sensor  balls 
have  been  modified  to  reduce  the  effect  of  the  C-130J 


vibration  environment  on  the  picture  clarity.  Although 
testing  of  the  newly  mounted  sensors  has  been  qualitative 
and  limited,  preliminary  results  suggest  the  sensors  can 
meet  system  threshold  requirements. 

-  The  sensor  operator  stick  controllers  on  the  MOP 
that  control  the  electro-optical/infrared  sensors  have 
been  modified  to  harden  them  against  radio  frequency 
interference  that  could  cause  un-commanded  sensor 
movements.  Again,  preliminary  results  from  limited 
testing  suggest  this  source  of  sensor  movement  may  have 
been  corrected. 

•  DT&E  identified  additional  problems  with  the  first  AC- 130J 
PSP  integration: 

-  Other  instances  of  un-commanded  sensor  movement 
occurred  that  were  not  attributable  to  radio  frequency 
interference  problems.  Investigation  identified  software 
mode  switching,  operator  error,  and  rapid  aircraft 
maneuvers,  which  exceeded  the  sensor’s  servo  rotation 
rates  as  causes.  The  program  has  recommended  crew 
workarounds  to  avoid  the  problem  while  it  investigates 
corrective  software  updates. 

-  The  computers  in  the  MOP  were  unable  to  keep  up  with 
the  maximum  computational  demands  placed  on  them. 
Operators  frequently  reported  components  of  the  mission 
management  software  locking  up  and  requiring  a  reset, 
and  in  some  cases,  the  hardware  itself  shuts  down  for 
thermal  protection.  Correcting  this  problem  requires  a 
modification  of  the  MOP  hardware  and  software.  In  the 
meantime,  operators  have  been  advised  to  operate  the 
MOP  in  a  degraded  configuration  with  limited  capabilities 
in  order  to  reduce  the  likelihood  of  a  shutdown,  although 
this  workaround  does  not  guarantee  system  stability.  The 
program  has  made  an  initial  modification  to  the  hardware 
and  continues  to  collect  MOP  performance  data  during  the 
crew  training  period. 

-  Four  other  software  issues  generated  Category  1 
deficiencies  that  may  not  be  corrected  until  the  software 
update  expected  on  the  Block  20  aircraft.  At  least  two  of 
the  deficiencies  could  hinder  timely  and  accurate  targeting 
of  weapons  because  of  data  latency. 

-  Operators  experienced  night  vision  compatibility  problems 
on  both  the  flight  deck  and  the  cargo  compartment.  A 
correction  to  the  flight  deck  problem  has  been  implemented 
and  will  be  tested  in  the  OUE  and  IOT&E. 

•  In  April  2015,  during  supplemental  flying  and  handling 
qualities  (F&HQ)  testing  brought  about  by  the  first  departure 
from  controlled  flight  in  February  2014,  the  aircraft 
experienced  a  second  departure  event.  The  recovery  maneuver 
overstressed  the  aircraft  and  the  damage  resulted  in  a  Class  A 
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mishap.  Aircraft  #1  return  to  flight  status  is  unknown. 
Preliminary  results  from  the  F&HQ  testing  suggest  there  are 
no  significant  differences  in  basic  C-130J  F&HQ  caused  by  the 
AC-130J  modifications. 

•  PSP  integration  problems,  along  with  the  addition  of 
high-priority  F&HQ  testing  after  the  first  departure  from 
controlled  flight  in  February  2014,  extended  the  DT&E 
schedule  into  July  2015,  delayed  end-to-end  testing  of 
weapons  employment  to  the  end  of  DT&E,  and  refocused  the 
program’s  efforts  on  a  minimum  essential  set  of  subsystem 
demonstrations.  As  a  result,  the  grounding  of  the  only  aircraft 
after  the  Class  A  mishap  effectively  ended  the  incomplete 
OA,  and  DT&E  concluded  prematurely  soon  after  that 
without  completing  the  full  intended  characterization  of 
system  capabilities.  This  left  the  planned  IOT&E  at  risk 

of  demonstrating  several  capabilities  for  the  first  time  and 
discovering  problems  that  are  more  appropriately  addressed 
during  DT&E. 

•  The  OA  conducted  on  the  Block  10  AC-130J  to  support  the 
Milestone  C  decision  and  ultimately  entrance  into  IOT&E 
in  1QFY16,  indicated  the  system  lacked  maturity  and  is  at 
risk  to  not  be  ready  for  IOT&E.  Survivability  was  largely 
unevaluated  due  to  flight  restrictions  in  place  and  deferred 
subsystem  integration  testing  in  DT&E.  Furthermore,  the 
acquisition  strategy  to  procure  at  most  three  Block  10  aircraft, 
then  focus  on  the  Block  20  aircraft  for  Initial  Operational 
Capability  (with  significant  capability  enhancements  and 
deficiency  corrections)  indicates  the  Block  20  configuration  is 
the  operationally  representative  configuration  for  IOT&E. 

•  The  program  recently  removed  full  operational  capability  of 
HELLFIRE  missiles  from  the  Block  20  requirement. 


•  A  limited  amount  of  live  fire  testing  against  representative 
targets  is  required  to  evaluate  the  lethality  of  the  AC-130J 
munitions  and  confirm  the  predictions  of  existing  lethality 
models.  These  tests  are  not  expected  to  increase  the  number  of 
live  shots  already  planned  for  operational  testing. 

•  Preliminary  assessment  of  the  AC-130J  survivability  against 
operationally  realistic  threats  will  begin  in  FY16  after 

the  completion  of  the  survivability  and  occupant  casualty 
analyses.  These  analyses  are  currently  informed  by  legacy 
aircraft  survivability  data.  DOT&E  will  supplement  them  with 
additional  AC- 13 0J  susceptibility  data,  after  the  completion  of 
relevant  developmental  and  operational  tests. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program  satisfied 
both  FY14  recommendations  regarding  preparation  for 
IOT&E,  which  has  now  been  converted  to  an  OUE  on  the 
Block  10  aircraft.  However,  the  Program  Office  still  needs  to 
provide  data  to  DOT&E  on  AC- 130  W  reliability,  which  could 
inform  the  AC-130J  evaluation. 

•  FY 15  Recommendations.  The  Program  Office  should: 

1.  Commit  to  the  full  accomplishment  of  lethality  testing 
by  the  end  of  IOT&E  and  work  with  the  18th  Flight  Test 
Squadron  to  make  resources  available  for  such  testing 
concurrent  with  operational  test  missions. 

2.  Develop  a  clear  and  stable  baseline  for  block  capability 
development,  test,  and  fielding,  and  brief  this  strategy  to 
DOT&E. 


AC-130J 
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Advanced  Extremely  High  Frequency  (AEHF) 
Satellite  Communications  System 


Executive  Summary 

•  Air  Force  Space  Command  declared  Initial  Operational 
Capability  (IOC)  on  the  Advanced  Extremely  High  Frequency 
(AEHF)  Satellite  Communications  (SATCOM)  system 

on  July  28,  2015,  and  provided  a  recommendation  to  the 
U.S.  Strategic  Command  (USSTRATCOM)  Commander  that 
the  system  was  ready  for  operations. 

•  From  October  27,  2014,  to  January  16,  2015,  the  Air 
Force  Test  and  Evaluation  Center  (AFOTEC)  conducted 
Multi-Service  Operational  Test  and  Evaluation  (MOT&E)  on 
the  AEHF  SATCOM  system.  The  MOT&E,  when  combined 
with  integrated  and  anti-jam  scintillation  testing,  was 
adequate  to  support  an  evaluation  of  the  system’s  operational 
effectiveness,  suitability,  and  survivability. 

•  The  AEHF  SATCOM  system  is  effective  in  providing 
survivable,  secure,  and  reliable  strategic  communications  at 
low  data  rates  (LDR)  and  tactical  communications  at  medium 
and  extended  data  rates  (MDR  and  XDR).  The  AEHF  system 
is  effective  in  maintaining  satellite  command  and  control  (C2), 
planning  communications,  and  managing  satellite  resources. 

•  DOT&E  submitted  an  MOT&E  report  to  Congress  in 
May  2015. 

•  The  AEHF  SATCOM  system  is  suitable  and  survivable. 

System 

•  The  AEHF  system  represents  the  second  generation  of 
Extremely  High  Frequency  SATCOM  capability  protected 
from  nuclear  effects  and  jamming  activities. 

•  The  AEHF  system  will  follow  the  Milstar  program  as 
the  protected  backbone  of  the  DOD’s  integrated  military 
SATCOM  architecture.  The  AEHF  constellation  of  four 
interconnected  satellites  is  expected  to  increase  system 
throughput  capacity  by  a  factor  of  10. 

•  The  AEHF  system  has  three  segments: 

-  Space  segment  -  an  integrated  constellation  of  Milstar  and 
AEHF  satellites 


-  Mission  Control  segment  -  includes  fixed  and  mobile 
telemetry,  tracking,  and  commanding  sites;  fixed  and 
transportable  communication  planning  elements;  the 
test  and  training  simulation  element;  and  the  operational 
support  and  sustainment  element 

-  Terminal  (or  User)  segment  -  includes  ground-fixed, 
ground-mobile,  man-portable,  transportable,  airborne, 
submarine,  and  shipboard  configurations 

Mission 

Combatant  Commanders  and  operational  forces  worldwide  use 

the  AEHF  system  to  provide  secure,  responsive,  and  survivable 

strategic  and  tactical  military  communications. 

Major  Contractors 

•  Lockheed  Martin  Space  Systems  -  Sunnyvale,  California 

•  Northrop  Grumman  -  Redondo  Beach,  California 


Activity 

•  AFOTEC  conducted  the  AEHF  MOT&E  from 

October  27,  2014,  to  January  16,  2015,  with  participation 
from  the  Navy’s  Commander,  Operational  Test  and  Evaluation 
Force,  the  Army  Test  and  Evaluation  Command,  and  the 
Marine  Corps  Operational  Test  and  Evaluation  Activity. 
AFOTEC  conducted  the  MOT&E  in  accordance  with  a 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 


-  The  Services  tested  the  AEHF  IOC  capabilities  to  include 
strategic  communications  at  LDR,  tactical  communications 
at  MDR  and  XDR,  satellite  C2,  communications  planning, 
and  satellite  resource  management. 

-  AFOTEC  augmented  the  MOT&E  data  with  anti-jam  and 
scintillation  testing  conducted  from  June  13,  2014,  through 
July  18,  2014,  and  integrated  testing  conducted  from 

July  15,  2014,  to  September  30,  2014. 


AEHF  313 


-  AFOTEC,  in  coordination  with  the  Massachusetts 
Institute  of  Technology’s  Lincoln  Laboratory,  performed 
threat  representative  jamming  of  AEHF’s  strategic 
communication  capability  and  the  nulling  capability  for 
tactical  communications.  Nulling  is  a  capability  that 
blocks  out  strong  jammer  signals  in  a  specific  geographical 
area  so  the  collocated  tactical  user’s  communications 
signals  can  be  heard. 

-  Three  Army  locations,  two  Marine  Corps  locations, 
two  Navy  destroyers  and  two  submarines,  the  E-4B 
National  Airborne  Operations  Center,  and  the  E-6B 
Airborne  Command  Post,  all  participated  in  the  test. 

-  AFOTEC  used  a  combination  of  live  operations, 
USSTRATCOM  exercise  “Global  Thunder,”  test-driven 
scenarios,  and  user  surveys  to  evaluate  strategic  and 
tactical  missions  of  the  AEHF  system.  The  test  team  tested 
various  tactical  networks,  including  a  representative  Army 
Corps  network,  a  Marine  Corps  Air-Ground  Task  Force 
network,  Navy  Time  Division  Multiple  Access  Interface 
Processor  and  broadcast  networks. 

•  DOT&E  wrote  and  submitted  an  MOT&E  report  to  Congress 
in  May  2015. 

•  Air  Force  Space  Command  declared  IOC  on  the  AEHF 
SATCOM  system  on  July  28,  2015,  and  provided  a 
recommendation  to  the  USSTRATCOM  Commander  that  the 
system  was  ready  for  operations. 

Assessment 

•  Operational  testing  of  the  AEHF  SATCOM  IOC  capability, 
when  combined  with  integrated  and  anti-jam  scintillation 
testing,  was  adequate  to  support  an  evaluation  of  the  system’s 
operational  effectiveness,  suitability,  and  survivability. 


•  The  AEHF  SATCOM  system  is  effective  in  providing 
survivable,  secure,  and  reliable  strategic  communications  at 
LDR  and  tactical  communications  at  MDR  and  XDR.  The 
AEHF  SATCOM  system  is  effective  in  maintaining  satellite 
C2,  planning  communications,  and  managing  satellite 
resources. 

•  The  Air  Force  deferred  the  XDR  capability  supporting 
strategic  forces  until  the  Final  Operational  Capability  is 
available,  as  these  networks  can  only  be  transitioned  from 
LDR  to  XDR  when  all  terminals  on  the  strategic  networks 
are  XDR-capable.  AFOTEC  will  operationally  test  XDR  for 
strategic  services  in  the  FOT&E,  tentatively  planned  for  FY21. 

•  The  AEHF  SATCOM  system  is  suitable.  The  AEHF  ground 
control  systems  meet  their  threshold  reliability  requirements. 
The  AEHF  satellites  and  ground  systems  meet  their 
dependability  requirements.  Maintenance  repair  times  for 
ground  control  systems  exceeded  threshold  requirements  in 
almost  all  areas  and  need  improvement;  however,  the  long 
maintenance  repair  times  did  not  adversely  affect  system 
dependability.  DOT&E  rated  human  factors,  system  usability, 
training,  and  documentation  favorably. 

•  The  AEHF  SATCOM  system  is  survivable  in  the  presence 
of  uplink  jamming,  scintillation,  and  external  cyber 
attack.  Strategic  communications  were  not  degraded  by  a 
threat-representative  jammer.  The  nuller  antenna  performance 
can  be  enhanced  with  operational  procedures. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has 
made  satisfactory  progress  on  addressing  all  previous 
recommendations . 

•  FY15  Recommendations.  None 
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AIM-120  Advanced  Medium-Range  Air-to-Air  Missile 

(AMRAAM) 


Executive  Summary 

•  The  Air  Force  and  Navy  completed  FOT&E  of  the  AIM-120D 
Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM)  in 
July  2014  and  fielded  the  system  in  January  2015. 

•  AIM-120D  System  Improvement  Program  (SIP)-l  operational 
test  activities  began  in  FY15  and  are  expected  to  be  completed 
in  FY16.  SIP-1  is  one  of  several  follow-on  programs  designed 
to  enhance  AIM- 120D  performance. 

•  The  Air  Force  continued  integrated  testing  on  AIM-120 
AMRAAM  Basic  Electronic  Protection  Improvement  Program 
(EPIP),  a  software  upgrade  to  AIM-120C3-C7  variants,  under 
a  separate  Basic  EPIP  Test  and  Evaluation  Master  Plan  that 
DOT&E  approved  in  April  2012. 

•  The  Air  Force  and  Navy  are  in  the  final  stages  of  test  planning 
to  conduct  cybersecurity  testing  for  all  variants  of  the 
AMRAAM  missile. 

System 

•  AMRAAM  is  a  radar-guided,  air-to-air  missile  with  capability 
in  both  the  beyond  visual-range  and  within  visual-range 
arenas.  A  single-launch  aircraft  can  engage  multiple  targets 
with  multiple  missiles  simultaneously  when  using  AMRAAM. 

•  F-15C/D/E,  F-16C/D,  F/A- 1 8C/D/E/F,  EA-18G,  and  F-22A 
aircraft  are  capable  of  employing  the  AMRAAM. 

•  The  AMRAAM  program  periodically  develops  and 
incorporates  phased  upgrades.  The  AMRAAM  Basic  EPIP  is 
a  software  upgrade  to  AIM-120C3-C7.  An  Advanced  EPIP 
software  upgrade  is  planned  to  add  additional  capabilities. 

•  A  SIP-2  upgrade  is  currently  in  planning. 

•  As  of  October  14,  2015,  Raytheon  has  delivered  a  total  of 
1,405  AIM- 120Ds. 


Mission 

•  The  Air  Force  and  Navy,  as  well  as  several  foreign  military 
forces,  use  various  versions  of  the  AIM- 120  AMRAAM  to 
shoot  down  enemy  aircraft. 

•  All  U.S.  fighter  aircraft  use  the  AMRAAM  as  the  primary, 
beyond  visual-range  air-to-air  weapon. 

Major  Contractors 

•  Raytheon  Missile  Systems  -  Tucson,  Arizona 

•  Rocket  Motor  Subcontractor:  Narnrno  (Nordic  Ammunition 
Group)  -  Raufoss,  Norway 


Activity 

•  The  Air  Force  and  Navy  conducted  all  testing  in  accordance 
with  the  DOT&E-approved  test  plan. 

AIM-120D 

•  AIM-120D  FOT&E,  which  the  Air  Force  and  Navy  completed 
in  July  2014,  consisted  of  multiple  live  missile  shots  and 
captive-carry  events.  The  missile  was  fielded  in  January  2015. 

•  The  Program  Office  conducted  SIP-1  integrated  testing  with 
two  live  missile  shots  in  March  and  May  2015.  Operational 
testing  for  SIP-1  began  in  January  2016. 

•  As  of  October  14,  2015,  Raytheon  has  delivered  a  total  of 
1,405  AIM-120Ds  for  the  Air  Force  and  Navy. 

AMRAAM  EPIP 

•  In  September  2014,  the  Air  Force  completed  Basic  EPIP 
operational  testing  for  AIM-120C-7  missiles. 


•  In  October  2014,  the  Air  Force  and  Navy  began  Basic  EPIP 
operational  testing  for  AIM-120C-3,  -4,  -5,  and  -6.  Reporting 
on  Basic  EPIP  operational  test  is  expected  to  be  completed  in 
FY16. 

AMRAAM  SIP 

•  The  Air  Force  identified  deficiencies  in  the  AIM-120D 
missile  performance  that  did  not  significantly  degrade  overall 
effectiveness.  The  Air  Force  and  Raytheon  Missile  Systems 
developed  solutions  for  specific  deficiencies  and  will  assess 
them  during  SIP- 1  testing. 

Cybersecurity 

•  The  Air  Force  and  Navy  are  in  the  final  stages  of  test  planning 
to  conduct  cybersecurity  testing  for  all  variants  of  the 
AMRAAM  missile. 
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Assessment 

•  AMRAAM  continues  to  be  operationally  effective  and 
suitable. 

•  Based  on  FY15  testing,  the  AIM-120D  SIP-1  missile  appears 
to  be  meeting  performance  and  reliability  requirements, 
although  a  final  assessment  will  not  be  available  until  after 
completion  of  SIP- 1  operational  testing  in  FY16. 

•  Missiles  equipped  with  Basic  EPIP  software  appear  to  be 
meeting  performance  requirements,  based  on  testing  to  date.  A 
final  assessment  of  Basic  EPIP  will  be  available  in  FY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
satisfactorily  addressed  the  previous  recommendations. 


•  FY 15  Recommendations.  The  Air  Force  should: 

1 .  Complete  SIP-2  and  Advanced  EPIP  operational  testing 
to  achieve  the  Services’  desired  mission  effectiveness 
improvements  for  AMRAAM. 

2.  Complete  cybersecurity  testing  for  all  variants  of  the 
AMRAAM  missile  in  accordance  with  the  August  1,  2014 
DOT&E  policy  memorandum. 
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Air  Force  Distributed  Common  Ground  System 

(AF  DCGS) 


Executive  Summary 

•  Air  Force  intelligence  units  use  the  AF  DCGS  to  produce 
intelligence  information  from  data  collected  by  a  variety  of 
sensors  on  the  U-2,  RQ-4  Global  Hawk,  MQ-1  Predator, 

MQ-9  Reaper,  MC-12,  and  other  Intelligence,  Surveillance, 
and  Reconnaissance  platforms. 

•  The  Air  Force  605th  Test  and  Evaluation  Squadron  completed 
Phase  1  of  a  three-phased  Force  Development  Evaluation 

on  the  Geospatial  Intelligence  (GEOINT)  Baseline  4.1  in 
June  2015.  While  GEOINT  Baseline  4.1  did  not  harm  any 
legacy  capabilities,  the  only  notable  functional  improvement 
was  the  ability  to  ingest  synthetic  aperture  radar  data  from 
Global  Hawk  Block  40. 

•  The  Air  Force  declared  Full  Operational  Capability  in  2009 
and  moved  the  program  from  the  development  phase  into 
the  sustainment  phase  despite  the  program’s  immaturity. 

In  August  2015,  the  Air  Force  restructured  AF  DCGS 
from  four  Acquisition  Category  (ACAT)  III  programs  into 
seven  ACAT  III  programs.  After  the  Air  Force  issued  the 
Acquisition  Decision  Memorandum  for  the  seven  programs, 
it  added  one  additional  ACAT  III  program  called  DCGS 
Reference  Imagery  Transition  (DRT),  making  a  total  of 
eight  ACAT  III  programs.  These  programs  lack  current 
and  accurate  test  and  evaluation,  systems  engineering,  and 
requirements  documentation. 

System 

•  The  AF  DCGS,  also  referred  to  as  the  AN/GSQ-272 
SENTINEL  weapon  system,  is  an  intelligence  enterprise 
system  that  is  composed  of  27-geographically  separated, 
networked  sites,  including  5  core  sites  across  the  globe. 

•  AF  DCGS  provides  hardware  and  software  tools  for  operators 
to  plan,  collect,  process,  exploit,  and  disseminate  Intelligence, 
Surveillance,  and  Reconnaissance  information.  The  DCGS 
Integration  Backbone  provides  the  framework  that  allows 
sharing  of  intelligence  services  and  data  via  web  services. 

•  The  Air  Force  declared  AF  DCGS  to  be  at  Full  Operational 
Capability  in  2009,  though  it  is  continuing  to  develop  the 
following  four  areas,  each  of  which  were  ACAT  III  programs: 
Signal  Intelligence  (SIGINT)  upgrades,  GEOINT  upgrades, 
Network  Communications,  and  Data  Links.  In  August  2015, 
the  Air  Force  restructured  AF  DCGS  from  four  ACAT  III 
programs  into  seven  ACAT  III  programs.  Subsequent  to  the 
decision  to  transition  to  seven  programs,  the  Air  Force  added 
one  additional  ACAT  III  program  called  DRT,  making  a  total 
of  eight  ACAT  III  programs.  Of  the  eight  programs,  only 

two  programs — GEOINT  Baseline  4.1  and  the  System  Release 
3.0 — have  been  operationally  tested. 

•  GEOINT  Baseline  4. 1  is  part  of  the  GEOINT  upgrades  portion 
of  AF  DCGS,  is  the  merger  of  Bulk  Release  10B  with  several 


AOC  -  Air  Operations  Center  JIOC  -  Joint  Intelligence  Operations  Center 
AWACS  -  Airborne  Warning  and  JSTARS  -  Joint  Surveillance  Target  Attack 
Control  System  Radar  System 

GH  -  Global  Hawk  JTF  -  Joint  Task  Force 

HQ  -  Headquarters  TARS  -  Tactical/Theater  Airborne 

JFC  -  Joint  Forces  Command  Reconnaissance  System 

deficiency  corrections,  and  the  integration  of  Airborne  Cueing 
and  Exploitation  System-Hyperspectral  (ACES-Hy)  and 
Global  Hawk  Block  40. 

•  System  Release  3.0  is  a  SIGINT  upgrade,  which  makes 
SIGINT  data  and  services  available  to  internal  and  external 
users,  improves  operations  with  the  Airborne  SIGINT  Payload 
low-band  sensor,  and  improves  processing,  exploitation,  and 
dissemination  for  high-band  sensors. 

Mission 

•  The  Air  Force  uses  AF  DCGS  for  Intelligence,  Surveillance, 
and  Reconnaissance  collection,  processing,  exploitation, 
analysis  and  dissemination. 

•  Air  Force  intelligence  units  use  the  AF  DCGS  to  produce 
intelligence  information  from  data  collected  by  a  variety  of 
sensors  on  the  U-2,  RQ-4  Global  Hawk,  MQ-1  Predator, 

MQ-9  Reaper,  MC-12,  and  other  Intelligence,  Surveillance, 
and  Reconnaissance  platforms. 

•  The  Air  Force  uses  AF  DCGS  to  connect  to  the  multi-Service 
DCGS  Integration  Backbone,  manage  requests  for  sensors, 
process  sensor  data,  exploit  sensor  data  from  multiple  sources, 
and  disseminate  intelligence  products. 

Major  Contractors 

•  Raytheon  -  Garland,  Texas 

•  Lockheed  Martin  -  Denver,  Colorado 

•  L-3  Communications  -  Greenville,  Texas 
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Activity 

•  From  November  through  December  2014,  and  in  May  2015, 
the  46th  Test  Squadron  conducted  developmental  and 
regression  testing  on  GEOINT  Baseline  4.1. 

•  In  June  2015,  the  605th  Test  and  Evaluation  Squadron 
conducted  Phase  1  of  the  three-phased  Force  Development 
Evaluation  at  DGS-5  to  assess  the  effectiveness  and  suitability 
of  GEOINT  Baseline  4. 1  in  support  of  a  fielding  decision 
following  all  three  test  phases. 

•  In  May  2015,  the  92nd  Information  Operations  Squadron 
conducted  a  cybersecurity  Cooperative  Vulnerability  and 
Penetration  Assessment  (CVPA)  of  System  Release  3.0 
(part  of  the  SIGINT  Upgrades  program)  at  DCGS  Ground 
Station  -  Experimental  (DGS-X). 

•  In  J  une  2015,  the  46th  Test  Squadron  conducted  a 
cybersecurity  CVPA  of  GEOINT  Baseline  4. 1  at  DGS-X. 

•  The  Air  Force  Program  Executive  Officer  for  Battle 
Management  signed  an  Acquisition  Decision  Memorandum 
on  August  6,  2015,  to  re-structure  the  program  into 

seven  Acquisition  Category  III  programs:  Sensor  Integration, 
GEOINT  Transformation,  GEOINT  Bulk  Release  4.1, 

SIGINT  Transformation,  System  Release  3.0,  Infrastructure 
Transformation,  and  Multi-Intelligence.  After  the  Acquisition 
Decision  Memorandum,  the  Air  Force  added  one  additional 
ACAT  III  program  called  DRT,  making  a  total  of  eight  ACAT 
III  programs.  These  programs  lack  current  and  accurate 
test  and  evaluation,  systems  engineering,  and  requirements 
documentation. 

•  The  eight  programs,  when  combined,  exceed  the  funding 
threshold  for  a  Major  Automated  Information  System  (MAIS) 
program.  The  Army  and  Navy  versions  of  DCGS  systems  are 
MAIS  programs. 

•  From  September  10  through  November  6,  2015,  the  Air 
Force  Operational  Test  and  Evaluation  Center  conducted 
an  Operational  Utility  Evaluation  on  System  Release  3.0. 
DOT&E  will  submit  a  report  on  the  results  of  testing  in  early 
2016. 

•  Testing  was  conducted  in  accordance  with  a 
DOT&E-approved  test  plan. 

Assessment 

•  The  program  continues  to  lack  current  requirements  and 
architecture  documents,  which  inhibit  the  tester’s  ability  to 
conduct  an  adequate  evaluation. 

•  The  program  lacks  a  rigorous  and  comprehensive  software 
problem  tracking  and  reporting  procedures.  The  Air  Force 
is  working  to  develop  and  implement  software  tracking  and 
reporting  via  the  AF  DCGS  Enterprise  Service  Desk. 

•  Multiple  DGS  users  created  ad-hoc  DCGS  analysis  and 
reporting  teams  to  perform  intelligence  fusion.  These  teams 
significantly  enhanced  situational  awareness  by  integrating 
multiple  intelligence  disciplines.  However,  the  Air  Force  did 
not  formally  resource  these  teams.  Thus,  the  teams  and  lack 
personnel,  doctrine,  training,  and  material  support. 


•  The  ability  of  GEOINT  Baseline  4. 1  to  ingest  synthetic 
aperture  radar  data  from  the  Global  Hawk  Block  40  is  the 
only  notable  functional  or  performance  gain  over  existing 
capability.  The  existing  capability  supports  high-  and 
medium-altitude  planning,  collection,  processing  and 
exploitation,  analysis  and  production,  and  dissemination. 

•  Full  motion  video  continues  to  have  problems  with  freezing 
and  degraded  images.  A  Category  1  Urgent  Test  Problem 
Report  has  been  open  since  June  2014  regarding  full  motion 
video  software.  Full  motion  video  analysts  continue  to  rely  on 
software  that  is  not  a  part  of  AF  DCGS,  and  that  the  Air  Force 
does  not  plan  to  continue  to  provide. 

•  Usability  concerns  remain  as  evidenced  in  the  low  System 
Usability  Scale  scores,  particularly  for  full  motion  video 
operators.  Mission  processes  are  complex  and  poorly 
documented. 

•  Training  is  lacking  in  quality  and  frequency,  and  maintenance 
documentation  and  positional  checklists  are  inadequate.  The 
training  survey  forms  indicate  some  operators  and  maintainers 
(19  of  143)  did  not  receive  any  GEOINT  Baseline  4. 1  training, 
while  others  (10  of  143)  specifically  said  they  received  training 
in  October  2014,  but  did  not  start  using  GEOINT  Baseline  4.1 
until  May  or  June  2015  with  no  refresher  training.  Analysts 
also  commented  that  documentation  and  checklists  were 
missing  key  information. 

•  Cybersecurity  testing  conducted  at  DGS-X  discovered 
vulnerabilities.  The  details  are  classified.  The  Air  Force  is 
working  on  implementing  fixes.  Cybersecurity  testing  of  an 
operational  site  is  planned  in  FY16-17. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
satisfactorily  addressed,  or  made  satisfactory  process  towards 
implementing,  two  of  five  previous  recommendations.  The 
three  previous  recommendations  still  outstanding  are: 

1.  Demonstrate  the  ability  of  AF  DCGS  to  operate  at 
anticipated  workload  levels.  This  requirement  is  still  being 
waived  by  the  Program  Office  prior  to  testing  since  much  of 
the  enterprise  is  not  yet  on  a  common  baseline. 

2.  Document  the  requirements  for  each  delivery  for  each 
of  the  four,  now  eight,  AF  DCGS  programs  and  conduct 
adequate  test  and  evaluation  based  on  a  risk  assessment 
in  accordance  with  DOT&E  guidelines.  Adequate  risk 
assessments  have  been  conducted  on  all  subsequent 
operational  tests;  however,  requirements  documentation  is 
still  lacking. 

3.  Submit  a  Test  and  Evaluation  Master  Plan  (TEMP)  for 
DOT&E  approval,  which  includes  an  accurate  description 
of  AF  DCGS  requirements,  architecture,  and  interfaces 
sufficient  to  justify  the  test  approach.  The  Program  Office 
is  making  progress  but  has  not  submitted  a  complete  draft 
TEMP. 
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FY15  Recommendations.  The  Air  Force  should: 

1.  Consider  formally  establishing  and  resourcing  the  DCGS 
analysis  and  reporting  team,  or  a  similar  all-source  fusion 
component,  in  each  DGS. 

2.  Develop  and  implement  a  software  change  request  process 
including  tracking  of  software  metrics  for  problems  open 
and  closed  by  severity  and  time. 

3.  Improve  training,  documentation,  and  checklists  prior  to  the 
next  phase  of  testing. 


4.  Document  all  known  cyber  vulnerabilities  in  a  plan  of 
action  and  milestones  and  track  the  progress. 

5.  Implement  the  planned  new  cybersecurity  demilitarized 
zone  architecture  prior  to  the  next  CVPA  at  an  operational 
site. 

6.  Document  and  review  operational  processes  and  simplify 
where  possible  to  support  training  and  operations. 
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Air  Operations  Center  -  Weapon  System  (AOC-WS) 


Executive  Summary 

•  The  Air  Operations  Center  -  Weapon  System  (AOC-WS)  10.1 
is  a  system-of-systems  that  contains  numerous  third-party 
software  applications,  including  the  Global  Command  and 
Control  System  -  Joint  (GCCS-J),  Theater  Battle  Management 
Core  Systems  -  Force  Level,  Master  Air  Attack  Plan  Toolkit, 
and  Joint  Automated  Deep  Operations  Coordination  System. 

•  The  Air  Force  tests  AOC-WS  10.1  during  a  three-phase 
Recurring  Event  (RE)  test  cycle,  which  includes  event-based 
test  periods  primarily  focused  on  software  upgrades.  The 
software  upgrades  and  associated  test  event  are  designated 
using  similar  terms;  for  example,  AOC-WS  10.1  RE13  is  the 
system  upgrade  tested  during  RE  13. 

-  Phase  1  developmental  testing  is  conducted  at  the 
Combined  Air  Operations  Center  -  Experimental 
(CAOC-X)  at  Joint  Base  Langley-Eustis,  Virginia. 

-  Phase  2  operational  testing  is  conducted  to  assess 
effectiveness  at  CAOC-X. 

-  Phase  3  operational  testing  is  conducted  at  a  fielded  site  to 
assess  suitability. 

•  In  October  2015,  the  Air  Force  delivered  its  final  report  on 
RE  13  that  included  the  results  of  Phase  3  operational  testing  at 
613  AOC,  Joint  Base  Pearl  Harbor-Hickam,  Hawaii. 

•  AOC-WS  10.1  RE  13  has  the  capability  to  produce  the  primary 
products  necessary  to  meet  the  established  AOC  battle  rhythm 
at  threshold  levels.  AOC-WS  10.1  RE13  demonstrated 
interoperability  with  other  mission-critical  systems. 

•  The  Air  Force  fully  assessed  cybersecurity  for  AOC-WS  10.1 
RE  13  and  identified  15  vulnerabilities  posing  significant  risk 
to  the  AOC  mission,  9  of  which  are  attributable  to  third-party 
applications  that  are  outside  the  control  of  the  AOC-WS 
Program  Office.  The  first-ever  Adversarial  Assessment  of  the 
AOC-WS  demonstrated  that  significant  cybersecurity  risk  to 
the  mission  exists. 

•  Following  the  completion  of  Phase  3  testing  at  613  AOC, 
there  was  a  single  Category  I  (CAT  I)  Urgent  functional 
deficiency.  Air  Combat  Command  conducted  an  analysis  of 
this  deficiency  and  has  deferred  the  implementation  of  the 
responsible  web-based  application  suite  during  RED  fielding 
until  the  Program  Office  has  remediated  the  deficiency. 

•  Air  Combat  Command  accepted  the  mission  risk  posed  by 
the  15  identified  cyber  vulnerabilities,  and  in  November 
2015  decided  to  field  AOC-WS  10.1  RED  to  meet  critical 
operational  needs,  while  maintaining  the  expectation  that 
the  AOC-WS  Program  Office  will  fix  unresolved  CAT  I 
deficiencies  in  an  expeditious  manner. 

•  AOC-WS  10.1  RED  is  built,  configured,  and  maintained 
at  operational  sites  with  the  assistance  of  a  Program  Office 
fielding  team.  The  site  leads  the  build  and  the  fielding  team 
augments  at  pre-planned  points  during  complex  segment 
installs.  Tier  1  help  desk  support  (AOC-WS  helpdesk  at  Joint 


Base  Langley-Eustis)  was  not  effective  during  the  test,  but 
Tier  2  (program  manager/vendor  support)  was  adequate  to 
support  the  system  fielding  and  operations  during  the  event. 
Subsequent  fielding  events  will  likely  depend  solely  on  Tier  2 
help  desk  support. 

System 

•  AOC-WS  is  the  senior  command  and  control  element  of  the 
U.S.  Air  Force’s  Theater  Air  Control  System  and  provides 
operational-level  command  and  control  of  air,  space,  and 
cyberspace  operations,  as  well  as  joint  and  combined  air, 
space,  and  cyberspace  operations.  Capabilities  include 
command  and  control  of  joint  theater  air  and  missile  defense, 
time-sensitive  targeting,  and  Intelligence,  Surveillance,  and 
Reconnaissance  management. 

•  The  AOC-WS  10.1  (AN/USQ-163  Falconer)  is  a  system 
of  systems  that  contains  numerous  software  applications 
developed  by  third  party  vendors  and  commercial  off-the-shelf 
products.  Each  third-party  system  integrated  into  the 
AOC-WS  provides  its  own  programmatic  documentation. 

•  The  AOC-WS  consists  of: 

-  Commercial  off-the-shelf  hardware 

-  Separate  third-party  software  applications,  including 
GCCS-J,  Theater  Battle  Management  Core 

Systems  -  Force  Level,  Master  Air  Attack  Plan  Toolkit, 
and  Joint  Automated  Deep  Operations  Coordination 
System,  from  which  the  AOC-WS  draws  its  capabilities 

-  Additional  third-party  systems  that  accept,  process, 
correlate,  and  fuse  command  and  control  data  from 
multiple  sources  and  share  them  through  multiple 
communications  systems 

•  AOC-WS  10.1  operates  on  several  different  local  area 
networks  (LANs),  including  Secret  Internet  Protocol  Router 
Network,  Joint  Worldwide  Intelligence  Communications 
System,  and  a  coalition  LAN,  when  required.  The  LANs 
connect  the  core  operating  system  and  primary  applications  to 
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joint  and  coalition  partners  supporting  the  applicable  areas  of 
operation.  Users  can  access  web-based  applications  through 
the  Defense  Information  Systems  Network. 

•  The  Air  Force  tests  AOC-WS  10.1  software  upgrades  during 
REs.  The  Air  Force  refers  to  each  software  upgrade  by  the  RE 
during  which  it  was  tested.  For  example,  AOC  WS  10.1  RE13 
is  the  software  upgrade  tested  during  RE13. 

•  The  future  AOC-WS  10.2  is  designed  to  deliver  a  modernized, 
integrated,  and  automated  approach  to  AOC  WS  operations. 

Mission 

The  Commander,  Air  Force  Forces,  or  the  Joint/Combined  Forces 

Air  Component  Commander  use  the  AOC-WS  to  exercise  control 

of  joint  (or  combined)  air  forces,  including  planning,  directing, 


and  assessing  air,  space,  and  cyberspace  operations  to  meet 
operational  objectives  and  guidance.  An  operational  AOC  is 
fundamental  in  enabling  centralized  command  and  decentralized 
execution  of  a  theater  air  campaign. 

Major  Contractors 

•  AOC-WS  10.1  Production  Center:  Jacobs  Technology 
Inc.,  Engineering  and  Technology  Acquisition  Support 
Services  -  Hampton,  Virginia 

•  AOC-WS  10.2  Modernization:  Northrop 
Grumman  -  Newport  News,  Virginia 


Activity 

•  The  Air  Force  typically  uses  a  three-phase  RE  test  cycle  for 
major  AOC  WS  10.1  upgrades,  along  with  lower-level  testing 
events,  to  sustain  interoperability  and  cybersecurity  and 
provide  low-risk  upgrades  to  third-party  systems  as  required. 

-  Phase  1  developmental  testing  is  conducted  at  CAOC-X 
Joint  Base  Langley-Eustis,  Virginia. 

-  Phase  2  operational  testing  is  conducted  at  CAOC-X  to 
assess  effectiveness. 

-  Phase  3  operational  testing  is  conducted  at  a  fielded  site  to 
assess  suitability. 

•  From  March  through  August  2015,  the  Air  Force  conducted 
operational  testing  of  AOC-WS  10.1  RE13  in  accordance 
with  the  DOT&E-approved  test  plans.  AOC-WS  10.1  RE13 
intended  to  deliver  new  operational  and  tactical  analysis 
capabilities,  upgraded  infrastructure,  updated  versions  of 
third-party  applications,  and  improved  system  cybersecurity 
posture. 

•  In  October  2015,  the  Air  Force  completed  its  report  on 
RE  13,  which  included  data  from  integrated  testing  (Phases 
1  and  2)  at  CAOC-X,  Joint  Base  Langley-Eustis,  Virginia, 
in  December  2014,  and  results  from  Phase  3  operational 
testing  at  613  AOC,  Joint  Base  Pearl  Harbor-Hickam,  Hawaii, 
from  March  through  August  2015.  Testing  at  the  613  AOC 
focused  on  three  areas:  assessing  the  ability  of  the  site  system 
administrators  to  correctly  install,  configure,  and  transition 
the  AOC  from  the  legacy  AOC-WS  10.1  RE12  version  to  the 
AOC-WS  10.1  RE13  capability;  validating  the  functional 
evaluation  data  obtained  during  developmental  test;  and 
assessing  operational  effectiveness  and  suitability  of  the 
upgraded  system. 

•  In  August  and  September  2015,  AOC-WS  10.2  completed 
the  first  of  two  scheduled  phases  of  developmental  testing  at 
CAOC-X.  The  severity  and  quantity  of  the  functional  and 
cybersecurity  deficiencies  identified  during  the  test  resulted  in 
the  Air  Force  issuing  a  cure  notice  to  the  prime  contractor. 


Assessment 

•  The  Air  Force  adequately  tested  AOC-WS  10.1  RE13 
through  a  combination  of  developmental  and  operational 
testing;  however,  there  were  significant  known  limitations  to 
Reliability,  Availability,  and  Maintainability  (RAM)  data. 

-  Testing  was  conducted  in  accordance  with  the 
DOT&E-approved  test  plans,  which  anticipated  the  lack  of 
RAM  data.  The  duration  and  nature  of  RE  13  test  events 
provided  insufficient  time  to  allow  DOT&E  to  assess  RAM 
under  operationally  realistic  system  usage. 

-  The  Air  Force  has  assessed  that  the  operational  tempo  in 
fielded  AOCs  precludes  the  level  of  manual  data  collection 
required  to  support  thorough  RAM  analysis.  However,  the 
Air  Force  plans  to  implement  a  technical  RAM  collection 
solution  in  the  modernization  increment,  AOC-WS  10.2, 
which  will  allow  DOT&E  to  conduct  thorough  analyses  for 
future  test  events. 

•  AOC-WS  10.1  RE  13  has  the  capability  to  produce  the  primary 
products  necessary  to  meet  the  established  AOC  battle  rhythm 
at  threshold  levels.  AOC-WS  10.1  RE13  demonstrated 
interoperability  with  other  mission-critical  systems. 

•  The  Air  Force  fully  assessed  AOC-WS  10.1  RE  13  for 
cybersecurity,  conducting  the  system’s  first-ever  Adversarial 
Assessment  using  a  DOD  cyber  Red  Team.  The  Cooperative 
Vulnerability  and  Penetration  Assessment  identified  1 1 
vulnerabilities,  9  of  which  are  attributable  to  third-party 
systems  that  could  pose  significant  risk  to  the  AOC-WS 
mission.  The  Adversarial  Assessment  identified  an  additional 
four  vulnerabilities  and  produced  associated  operational  effects 
that  demonstrate  significant  risk  to  the  mission  exists. 

•  The  Program  Office  successfully  closed  six  of  the  seven 
functional  CAT  I  deficiencies  found  during  the  RE  13  test 
events.  The  single  open  RE  13  CAT  I  functional  deficiency 
is  a  result  of  implementing  a  suite  of  web-based  applications 
for  submitting,  managing,  and  querying  air  mission  reports; 
Intelligence,  Surveillance,  and  Reconnaissance  post-mission 
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summaries;  and  electronic  warfare  mission  requests  and 
reporting. 

-  This  deficiency  affects  operational  suitability  by  requiring 
communications  support  personnel  to  create  and  maintain 
accounts  for  thousands  of  external  users  across  the 
Services. 

-  This  deficiency  could  also  affect  operational  effectiveness 
if  personnel  deploying  to  a  theater  were  unable  to  establish 
accounts  in  a  timely  manner,  or  if  aircrews  transiting  the 
theater  without  accounts  were  not  able  to  submit  mission 
reports.  Both  scenarios  would  negatively  affect  the  ability 
of  an  AOC  to  conduct  intelligence  analysis  required  to 
accomplish  mission  tasks. 

-  Following  the  completion  of  Phase  3  testing  at  613  AOC, 
Air  Combat  Command  conducted  an  analysis  of  this 
deficiency.  The  command  has  deferred  the  implementation 
of  the  web-based  application  suite  during  RE  13  fielding 
until  the  Program  Office  has  remediated  the  deficiency. 

•  Air  Combat  Command  accepted  the  mission  risk  posed  by 
the  15  identified  cyber  vulnerabilities,  and  decided  to  field 
AOC-WS  10.1  RE13  to  meet  critical  operational  needs,  while 
maintaining  the  expectation  that  the  AOC-WS  Program  Office 
will  fix  unresolved  CAT  1  deficiencies  in  an  expeditious 
manner. 

•  AOC-WS  10.1  RE  13  can  be  built,  configured,  and  maintained 
adequately  at  operational  sites  with  Program  Office-provided 
support  during  specific  complex  installation  segments.  Tier  1 
help  desk  support  was  not  effective  for  build  support,  but  Tier 
2  was  adequate  to  support  the  system  fielding  and  operations 
during  the  event.  Subsequent  fielding  events  will  likely 
depend  solely  on  Tier  2  help  desk  support. 

•  The  key  to  successful  testing  and  fielding  of  AOC-WS  10.1 
continues  to  be  close  collaboration  between  the  AOC-WS 


Program  Office  and  the  providers  of  third-party  applications 
to  ensure  those  applications  meet  the  operational  and 
cybersecurity  needs  of  the  AOCs.  Early  AOC-WS  tester 
involvement  in  third-party  testing  continues  to  be  necessary  to 
identify  critical  problems  for  early  corrective  action. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has 
fully  addressed  one  previous  recommendation  and  has  made 
progress  towards  two  previous  recommendations.  The  Air 
Force  still  needs  to  require  AOC  sites  to  collect  and  report 
RAM  data  to  the  Program  Office.  The  Air  Force  has  assessed 
that  the  operational  tempo  in  fielded  AOCs  precludes  the 
level  of  manual  data  collection  required  to  support  thorough 
RAM  analysis.  Therefore,  the  Air  Force  plans  to  implement 
a  technical  RAM  collection  solution  in  the  modernization 
increment,  AOC-WS  10.2,  which  will  allow  DOT&E  to 
conduct  thorough  analyses  for  future  test  events. 

•  FY15  Recommendations.  The  Air  Force  should: 

1.  Close  identified  cybersecurity  vulnerabilities  in  cooperation 
with  third-party  system  providers  to  mitigate  risk  to  the 
AOC  mission. 

2.  Improve  dynamic  cyber  defensive  capabilities  focusing  on 
detecting  and  responding  to  cyber  adversary  attacks  against 
the  AOC-WS. 

3.  Reassess  the  help  desk-enabling  concept  to  determine 
whether  Tier  1  help  desk  personnel  can  be  sufficiently 
trained  to  support  the  build  process.  For  future  build 
support,  the  Air  Force  should  consider  merging  Tier  1  and 
Tier  2  functionality. 
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CV-22  Osprey 


Executive  Summary 

•  Air  Force  Special  Operations  Command  (AFSOC)  procured 
a  removable  armor  package  for  the  CV-22  in  response  to 

an  urgent  need  request.  LFT&E  of  the  armor  package 
demonstrated  improved  force  protection  of  the  CV-22  platform 
against  selected  small  arms  threats. 

•  Preliminary  assessment  of  integrated  testing  of  Suite  of 
Integrated  Radio  Frequency  Countermeasures  (SIRFC) 
software  version  8.02  did  not  demonstrate  aircraft 
survivability  improvements  against  selected  radio  frequency 
threats.  SIRFC  performance  is  consistent  with  previous  test 
results  and  therefore  does  not  meet  survivability  requirements 
against  some  threat  systems. 

•  The  Program  Office  addressed  key  deficiencies  revealed 
during  the  developmental  testing  of  the  next  Tactical  Software 
Suite,  new  mission  computer,  and  a  new  Color  Helmet 
Mounted  Display.  Operational  testing  of  these  systems  is 
scheduled  for  1QFY16. 

System 

•  The  CV-22  is  the  AFSOC  variant  of  the  V-22.  It  succeeded 
Special  Operations  Forces  MH-53  helicopters  in  2008. 

The  tilt-rotor  design  provides  the  speed  and  range  of  a 
conventional  fixed-wing  aircraft  and  vertical  take-off  and 
landing  capabilities  of  a  helicopter. 

•  The  CV-22  has  terrain-following/terrain-avoidance  radar, 
an  advanced  multi-frequency  radio  communication  suite, 
an  integrated  electronic  defense  suite  and  aerial  refueling 
capability,  allowing  it  to  augment  the  AFSOC  MC-130  fleet. 

•  The  CV-22  electronic  defensive  suite  includes  the  SIRFC 
and  the  Directional  Infrared  Countermeasures  (D1RCM) 
System  with  the  AAR-54  Missile  Warning  Sensor,  Small 
Laser  Transmitter  Assembly  jammer  and  the  ALE-47 
Countermeasure  System  capable  of  dispensing  both  flares  and 
chaff.  The  Dedicated  Electronic  Warfare  Display  provides 

an  integrated  threat  picture  to  the  crews  from  SIRFC  and 
D1RCM. 

•  The  CV-22  can  carry  18  combat-ready  Special  Operators 
538  nautical  miles  and  return.  It  can  self-deploy  up  to 
2,100  nautical  miles  with  one  aerial  refueling. 

•  In  response  to  CV-22  damage  and  passenger  casualties  in  a 
December  2013  combat  incident  in  South  Sudan,  AFSOC 


submitted  an  urgent  need  request  for  a  removable  armor 
package  to  protect  passengers  from  small  arms  threats. 
AFSOC,  in  collaboration  with  the  V-22  Joint  Program  Office 
(PMA-275),  established  requirements,  and  awarded  the 
Advanced  Ballistic  Stopping  System  (ABSS)  contract  to  The 
Protective  Group,  Inc.  The  final  ABSS  kit  solution  weighs 
825  pounds.  The  vendor  delivered  16  ABSS  kits  to  AFSOC  in 
September  2014. 

•  Bell-Boeing  delivered  45  of  50  purchased  aircraft;  42  are 
operational  and  3  of  the  remaining  50  are  in  storage  awaiting 
operational  testing  that  is  on  hold,  pending  completion  of 
deficiency  report  corrective  actions  for  Mission  Computer 
Obsolescence  Initiative  and  the  Color  Helmet  Mounted 
Display.  The  final  two  aircraft  are  expected  to  be  delivered  by 
the  end  of  2016. 

Mission 

Commanders  will  employ  AFSOC  squadrons  equipped  with  the 
CV-22  to  provide  high  speed,  long-range  insertion  and  extraction 
of  Special  Operations  Forces  to  and  from  high-threat  objectives. 

Major  Contractors 

•  Bell-Boeing  Joint  Venture: 

-  Bell  Helicopter  -  Amarillo,  Texas 

-  The  Boeing  Company  -  Ridley  Township,  Pennsylvania 

•  The  Protective  Group,  Inc.  -  Miami  Lakes,  Florida 


Activity 

•  In  2014,  Naval  Air  Systems  Command  completed  expedited 
live  fire  testing  of  the  ABSS  armor  and  evaluated  its 
performance  against  the  specification  threat.  To  help  meet  the 
urgent  need  request  for  new  armor,  DOT&E  did  not  require 


approval  authority  of  the  ABSS  armor  live  fire  test  plan. 
Instead,  DOT&E  reviewed  the  test  plan  and  ensured  the  test 
was  conducted  in  accordance  with  the  agreed  upon  test  plan. 
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•  AFSOC  conducted  operational  testing  in  September  2014  to 
evaluate  the  ability  of  the  unit  to  remove  and  install  the  armor 
and  the  ability  of  the  crew  to  perform  their  normal  mission 
with  the  armor  installed. 

•  AFSOC  completed  a  fraction  of  the  upgraded  SIRFC  software 
version  8.02  tests  in  February  through  March  2015  at  China 
Lake  and  Nevada  Test  and  Training  Range  to  address  CV-22 
SIRFC  active  counter-measure  deficiencies.  Remaining 
testing  of  the  SIRFC  software  was  completed  in  October  2016. 

•  The  Navy  has  continued  with  developmental  testing  on  the 
new  Tactical  Software  Suite  (TSS)  20.2.02,  which  has  included 
testing  of  a  new  aircraft  mission  computer  needed  to  address 
obsolescence  problems.  Operational  testing  of  the  TSS 
20.2.02  and  the  Color  Helmet  Mounted  Display  is  scheduled 
for  1QFY16. 

•  AFSOC  also  plans  to  conduct  a  Cooperative  Vulnerability 
and  Penetration  Assessment  of  cybersecurity  protections  and 
vulnerabilities  in  the  first  half  of  FY16. 

Assessment 

•  DOT&E  assessed  the  conduct  of  the  ABSS  armor  test  against 
the  specification  threat  was  adequate. 

•  ABSS  armor  met  the  ballistic  protection  requirements  for 
a  selected  range  of  engagement  conditions.  ABSS  armor 
effectiveness  against  other  operationally  realistic  threats  was 
not  assessed. 

•  Operational  testing  demonstrated  that  the  CV-22  aircrew  and 
maintenance  personnel  were  able  to  perform  their  mission 
with  the  ABSS  installed.  Testing  also  demonstrated  that  the 
maintenance  personnel  would  require  approximately  8  hours  to 
install  (or  remove)  ABSS,  prior  to  the  mission.  Due  to  added 
weight,  installed  armor  imposes  a  limited  penalty  in  combat 
range  and  in  a  number  of  combat-ready  Special  Operators  on  a 
mission. 

•  Preliminary  data  analyses  suggest  that  the  active 
counter-measure  component  of  the  SIRFC  8.02  system  did  not 
address  the  subsystem  deficiencies.  Consistent  with  previous 
results,  the  subsystem  does  not  meet  most  survivability 
requirements. 


•  The  AFSOC  identified  four  deficiencies  in  the  TSS  20.2.02 
developmental  testing  during  FY14:  (1)  the  mission  computer 
randomly  dropped  map  symbols  after  the  map  symbology 
memory  data  reached  its  capacity;  (2)  vibrations,  associated 
with  certain  flight  conditions,  degraded  the  readability  of  the 
Color  Helmet  Mounted  Display;  (3)  the  new  data  transfer  unit 
for  navigation,  communication,  and  threat  data  created  the 
possibility  for  data  spillage  between  classified  and  unclassified 
systems;  and  (4)  the  Intelligence  Broadcast  Receiver,  which 
provides  near  real-time  updates  on  threat  and  survivor  location 
throughout  a  mission,  occasionally  erased  threat  data  during 
flight,  even  though  erasure  was  not  commanded.  With  the 
exception  of  the  dropped  map  symbols  issue,  which  is  too 
extensive  to  make  the  current  TSS  20.2.02  release  and  will 

be  addressed  in  future  software  release,  the  Program  Office 
addressed  all  key  deficiencies.  These  activities,  in  part, 
delayed  entry  into  operational  testing  until  FY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Services  have  been 
working  on  addressing  the  two  FY13  recommendations.  The 
Navy  completed  developmental  testing  of  the  TSS  20.2.02  to 
attempt  to  address  the  SIRFC  deficiencies  while  AFSOC  has 
been  working  on  test  plans  to  start  the  radio-communication 
test  in  the  context  of  end-to-end  operational  missions  in  a 
variety  of  operational  and  atmospheric  conditions  in  FY16. 

•  FY15  Recommendations. 

1.  The  Navy  should  continue  to  address  deficiencies  in  SIRFC 
active  counter-measure  performance  and  AFSOC  should 
verify  deficiency  correction  in  future  operational  testing. 

2.  DOT&E  funded  a  joint  live  fire  program  to  assess  the 
performance  of  ABSS  against  threats  not  assessed  in  the 
CV-22  ABSS  LFT&E  program.  The  CV-22  Program 
Office  and  AFSOC  should  review  these  data  to  account 
for  the  effectiveness  of  the  ABSS  against  these  additional, 
operationally  realistic  threats  and  to  adjust  the  tactics, 
techniques,  and  procedures,  as  needed. 
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Defense  Enterprise  Accounting  and  Management 

System  (DEAMS) 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  an  IOT&E  of  the  Defense  Enterprise 
Accounting  and  Management  System  Increment  1  (DEAMS 
Inc  1)  between  October  1,  2014,  and  May  29,  2015,  at  seven 
bases  that  included  three  Air  Force  major  commands,  three 
U.S.  Combatant  Commands,  and  the  Defense  Finance  and 
Accounting  Service  (DFAS).  The  Program  Executive  Officer 
paused  testing  twice  to  allow  users  to  clear  a  backlog  of  failed 
or  delayed  transactions,  which  were  affecting  operational 
performance. 

•  Army  and  Air  Force  cybersecurity  teams  conducted  a 
Cooperative  Vulnerability  and  Penetration  Assessment  and 
Adversarial  Assessment.  The  Joint  Interoperability  Test 
Command  (JITC)  evaluated  the  system’s  interoperability  with 
external  systems. 

•  DEAMS  Inc  1  was  not  operationally  effective,  not 
operationally  suitable,  and  not  survivable  against  the 
cybersecurity  threat.  In  preparation  for  the  end-of-year  close 
for  FY15,  a  portion  of  software  release  3  was  fielded  to 
support  end-of-year  accounting  closeouts,  which  adversely 
affected  operational  performance  during  the  IOT&E.  DEAMS 
Inc  1  did  not  effectively  perform  several  critical  accounting 
and  management  tasks,  four  of  which  were  Key  Performance 
Parameters  (KPPs). 

•  DEAMS  Inc  1  is  not  survivable  against  the  cybersecurity 
threat.  Cybersecurity  testing  during  the  IOT&E  showed  that 
the  system  did  not  adequately  protect  its  financial  management 
information  and  neither  operators  nor  administrators  were 
able  to  detect  cybersecurity  intrusions.  In  February  2014, 

the  177th  Aggressor  Squadron  conducted  a  Cyber  Economic 
Vulnerability  Penetration  Assessment  of  DEAMS  Inc  1  that 
revealed  serious  cyber  vulnerabilities.  DOT&E  made  several 
recommendations  for  mitigating  these  cyber  vulnerabilities  in 
its  classified  annex  to  the  DEAMS  Operational  Assessment 
report  dated  June  2014,  but  during  the  IOT&E,  the  177th 
found  cyber  vulnerabilities  similar  to  those  found  in  2014. 

System 

•  DEAMS  Inc  1  is  a  Major  Automated  Information  System  that 
uses  commercial  off-the-shelf  Enterprise  Resource  Planning 
software  to  provide  accounting  and  management  services. 

•  The  DEAMS  Inc  1  Program  Office  is  following  an 
evolutionary  acquisition  strategy  that  adds  additional 
capabilities  and  users  incrementally.  There  are 

seven  scheduled  releases.  Release  3,  currently  fielded,  had  at 
least  5,000  users  at  the  time  of  the  IOT&E  and  is  expected  to 
have  approximately  23,500  users  worldwide  by  2017. 


DEAMS 

General  Ledger 
Funds  Control 

Accounts  payable/receivable 
Cost  Accounting/Management 
Billings/Collections 
Commitments/Obligations 
Purchasing  &  Receipts 
Revenue 

Analysis/Decision  Support 
Budget  Formulation 
Budget  Execution 
Cost  Modeling 


•  DEAMS  Inc  1  is  intended  to  improve  financial  accountability 
by  providing  a  single,  standard,  automated  financial 
management  system  that  is  compliant  with  the  Chief  Financial 
Officer’s  Act  of  1990  and  other  mandates.  DEAMS  Inc  1 
performs  the  following  core  accounting  functions: 

-  Core  Financial  System  Management 

-  General  Ledger  Management 

-  Funds  Management 

-  Payment  Management 

-  Receivable  Management 

-  Cost  Management 

-  Reporting 

•  DEAMS  Inc  1  interfaces  with  approximately  40  other  systems 
that  provide  travel,  payroll,  disbursing,  transportation, 
logistics,  acquisition,  and  accounting  support. 

•  DEAMS  Inc  1  supports  financial  management  requirements 
in  the  Federal  Financial  Management  Improvement  Act  and 
DOD  Business  Enterprise  Architecture;  therefore,  it  is  subject 
to  the  2010  National  Defense  Authorization  Act  requirement 
to  be  auditable  by  2017. 

Mission 

Air  Force  financial  managers  and  tenant  organizations  use 
DEAMS  Inc  1  to  do  the  following  across  the  Air  Force, 

U.S.  Transportation  Command,  and  other  U.S.  component 
commands: 

•  Compile  and  share  accurate,  up-to-the-minute  financial 
management  data  and  information. 

•  Satisfy  Congressional  and  DOD  requirements  for  auditing  of 
funds,  standardizing  of  financial  ledgers,  timely  reporting,  and 
reduction  of  costly  rework. 

Major  Contractor 

Accenture  Federal  Services  -  Dayton,  Ohio 


Provide  Financial 
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Provide  Budget 
Formulation,  Funds 
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Cost  Modeling 


Manage  DoD 
Appropriated  & 
Working  Capital 
Funds 


Process  Budgetary, 
Accounting,  & 
Vendor  Pay 
Transactions 
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Activity 

•  AFOTEC  conducted  an  IOT&E  of  DEAMS  Inc  1  between 
October  1,  2014,  and  May  29,  2015,  at  seven  Air  Force 
bases  that  included  three  Air  Force  major  commands,  three 
U.S.  Combatant  Commands,  and  DFAS  at  Limestone,  Maine. 

•  The  IOT&E  did  not  run  continuously  during  the 
8-month  period.  The  Program  Executive  Officer  paused 
testing  twice  with  DOT&E  approval  to  allow  Air  Force 
financial  managers  and  DFAS  to  clear  a  backlog  of  failed  or 
delayed  transactions,  affecting  operational  performance  of 
DEAMS  Inc  1. 

•  In  conjunction  with  the  IOT&E,  the  Army  Research 
Laboratory  in  White  Sands  Missile  Range,  New  Mexico, 
conducted  a  cybersecurity  Cooperative  Vulnerability  and 
Penetration  Assessment  at  Maxwell  AFB  -  Gunter  Annex, 
Alabama.  The  177th  Information  Aggressor  Squadron 
followed  this  up  with  an  Adversarial  Assessment  at  McConnell 
AFB,  Kansas. 

•  In  August  2015,  DOT&E  submitted  an  IOT&E  report  with  a 
classified  annex  detailing  the  results  of  the  IOT&E  testing. 

•  JITC  evaluated  DEAMS  Inc  1  interoperability  with  external 
systems. 

•  AFOTEC  conducted  the  IOT&E  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  the 
test  plan. 

Assessment 

•  DEAMS  Inc  1  demonstrated  good  progress  during  the 
second  operational  assessment  in  2014,  but  a  portion  of 
software  release  3  was  fielded  between  the  second  operational 
assessment  and  IOT&E  to  support  end-of-year  accounting 
closeouts,  which  adversely  affected  operational  performance 
during  the  IOT&E. 

•  DEAMS  Inc  1  was  not  operationally  effective.  DEAMS  Inc 
1  did  not  effectively  perform  several  critical  accounting  and 
management  tasks,  four  of  which  were  KPPs. 

-  DEAMS  Inc  1  failed  to  correctly  balance  available 
funds,  close  end-of-year  accounts  within  the  time  frame 
prescribed  by  the  Air  Force,  meet  the  95  percent  thresholds 
for  balancing  end-of-quarter  and  end-of-month  accounts, 
and  record  transactions  in  a  timely  manner. 

-  Poor  DEAMS  performance  resulted  in  an  increase  in 
late  penalty  payments  to  $465.7  4  per  $1  Million  in 
January  2015,  which  was  nearly  10  times  the  Air  Force’s 
FY 15  goal  of  $49.00  per$l  Million. 

-  DEAMS  Inc  1  failed  to  process  some  transactions  during 
IOT&E,  resulting  in  a  backlog  of  failed  or  delayed 
transactions.  The  DFAS  users’  workload  increased  as 
they  tried  to  reduce  the  backlog.  DEAMS  usability  will 
continue  to  be  degraded  until  the  backlog  is  reduced. 

•  Problems  with  DEAMS  Inc  l’s  reporting  tool  (Discoverer) 
have  been  noted  during  the  IOT&E  and  during  previous 
DEAMS  Inc  1  operational  testing.  Enhancements  to  DEAMS 
Inc  1  intended  to  correct  reporting  defects,  including  the 
Oracle  Business  Intelligence  Enterprise  Edition  (OBIEE), 
were  not  fielded  for  the  IOT&E,  even  though  OBIEE  was 


included  in  release  3  according  to  the  DEAMS  acquisition 
strategy.  While  OBIEE  was  not  ready  and  not  fielded,  when 
it  is  fielded,  it  is  expected  to  provide  improved  reporting. 
IOT&E  survey  results  indicated  that  48  percent  of  users  did 
not  use  DEAMS  Inc  1  for  status  of  funds  reporting,  budget 
analysis  and  planning,  or  decision  support,  which  meant  that 
for  this  set  of  users,  DEAMS  Inc  1  reporting  functionality  was 
not  being  used  for  financial  decision-making  as  intended. 

•  DEAMS  Inc  1  is  not  operationally  suitable.  DEAMS  Inc  1 
continues  to  exhibit  problems  with  software  reliability  growth 
as  measured  by  an  increase  in  the  number  of  high-severity 
deficiencies,  including  many  that  had  remained  unresolved  for 
240  days  or  longer.  Configuration  management  procedures 
regarding  regression  testing  of  resolved  defects  were  either 
not  followed  by  the  Program  Office  or  were  inadequate  to 
ensure  that  the  defects  were  actually  corrected  in  the  software. 
DEAMS  Inc  1  also  did  not  meet  Net-Ready  KPP  requirements 
for  the  exchange  of  critical  information  as  reported  by  JITC. 
The  results  of  the  interoperability  assessment  demonstrated 
that  5  of  the  22  available  critical  interfaces  did  not  meet  the 
required  information  exchange  requirements. 

•  Although  DEAMS  Inc  1  training  demonstrated  gradual 
improvement  since  the  second  operational  assessment,  the 
training  did  not  prepare  most  users  to  employ  DEAMS  Inc 
1  effectively  and  did  not  provide  knowledge  to  new  users 
to  adequately  perform  necessary  accounting  and  reporting 
tasks.  A  long  time  gap  between  initial  training  and  IOT&E 
degraded  user  proficiency,  as  much  of  what  was  learned  had 
been  forgotten.  The  training  needs  to  align  more  closely 
with  deployments  rather  than  being  given  months  in  advance. 
Training  alignment  could  prove  a  challenge  for  presently 
scheduled  deployments  to  many  thousands  of  new  users  and 
scores  of  new  bases  over  the  next  year. 

•  In  the  June  2014  operational  assessment  report,  DOT&E 
recommended  that  the  Air  Force  modify  DEAMS  Inc  1 
user  training  to  focus  more  on  functional  understanding 
the  DEAMS  Inc  1  general  ledger  environment,  rather  than 
on  navigation  within  the  system.  However,  during  the 
IOT&E,  users  indicated  that  the  training  focused  on  how  to 
navigate  within  DEAMS  Inc  1  rather  than  how  to  perform 
missions  in  the  DEAMS  Inc  1  environment.  Additionally, 
the  business  process  changes  that  accompanied  DEAMS  Inc 
1  implementation  were  not  intuitive  to  the  users  based  upon 
their  experience  with  legacy  systems.  Accordingly,  users  were 
not  readily  able  to  accomplish  their  tasks. 

•  Training  that  focuses  on  providing  functional  understanding 
will  improve  the  suitability  of  DEAMS  Inc  1 . 

•  DEAMS  Inc  1  is  not  survivable  against  the  cybersecurity 
threat.  In  February  2014,  the  177th  Aggressor  Squadron 
conducted  a  Cyber  Economic  Vulnerability  Penetration 
Assessment  of  DEAMS  Inc  1  that  revealed  serious  cyber 
vulnerabilities.  Cybersecurity  testing  showed  that  the 
system  did  not  adequately  protect  its  financial  management 
information  and  neither  operators  nor  administrators  were 
able  to  detect  cybersecurity  intrusions.  DOT&E  made  several 
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recommendations  for  mitigating  these  vulnerabilities  in  its 
classified  annex  to  the  DEAMS  Operational  Assessment  report 
dated  June  2014,  but  during  the  IOT&E,  the  177th  found 
vulnerabilities  similar  to  those  found  in  2014. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Program  Office  did 
not  implement  FY14  recommendations  successfully  and  still 
needs  to: 

1 .  Correct  balance  accuracy  defects  to  meet  KPP  requirements 
and  to  demonstrate  progress  towards  DEAMS  Inc  1 
achieving  of  full  auditability. 

2.  Address  cybersecurity  recommendations  provided  in 
DOT&E’s  classified  annex  to  the  DEAMS  Operational 
Assessment  report. 

•  FY15  Recommendations.  The  DEAMS  program  manager 
should: 

1 .  Identify  and  implement  processes,  procedures,  and  software 
improvements  to  clear  the  transaction  backlog  to  fix  the  lag 


time  between  transaction  and  posting  and  ensure  accurate 
and  timely  reporting. 

2.  Work  with  the  DFAS  to  identify  the  root  causes  of 
imbalances  between  DEAMS  Inc  1  and  Treasury  and 
change  policies  and  procedures  or  implement  software 
improvements  to  prevent  further  imbalances. 

3.  Conduct  regression  testing  to  improve  DEAMS  Inc  1 
performance  and  identify  potential  interface  problems 
before  fielding  software  updates  and  releases. 

4.  Provide  DEAMS  Inc  1  training  that  prepares  users  to 
effectively  employ  DEAMS  Inc  1  upon  fielding. 

5.  Work  with  AFOTEC  to  conduct  follow-on  operational 
testing  to  verify  that  the  deficiencies  have  been  corrected 
and  that  the  new  reporting  tool  is  operationally  effective, 
suitable,  and  cyber-secure,  once  corrections  have  been  made 
and  a  new  reporting  tool  has  been  fielded. 
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F-22A  Advanced  Tactical  Fighter 


Executive  Summary 

•  F-22A  Increment  3.2A  is  a  software-only  modernization  effort 
integrating  Link  16  Receive,  enhanced  combat  identification, 
and  enhanced  electronic  protection  capabilities.  Increment 
3.2A  developmental  testing  completed  in  early  FY15; 
however,  several  software  stability,  radar,  and  Link  16 
datalink  performance  shortfalls  remained  unresolved  during 
developmental  testing  and  were  carried  forward  into  FY15 
FOT&E. 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  FOT&E  of  Increment  3. 2  A  from 
November  2014,  through  May  2015.  FOT&E  results 
demonstrated  that  the  Operational  Flight  Program  (OFP) 
software  suite  is  operationally  effective;  however,  shortfalls 
carried  forward  from  developmental  testing  and  re-discovered 
during  FOT&E  detract  from  overall  operational  capability. 

•  Increment  3.2A  FOT&E  results  further  indicated  the  OFP 
software  suite  is  not  operationally  suitable  as  it  did  not 
meet  the  Air  Force  threshold  Mean  Time  Between  Avionics 
Anomaly  requirement,  and  there  is  a  high  probability  that 
pilots  may  not  be  able  to  complete  combat  missions  without 
having  to  reset  or  restart  avionics  subsystems  during  flight. 

•  F-22A  Increment  3.2B  is  a  separate  Major  Defense  Acquisition 
Program  modernization  effort  intended  to  integrate  AIM  120D 
and  AIM-9X  missile  systems  and  provide  additional  electronic 
protection  enhancements  and  improved  emitter  geolocation 
capability.  Ongoing  developmental  testing  experienced 
several  delays  due  to  additional  unplanned  regression  testing 
for  Increment  3.2A  and  Update  5  OFP  efforts,  and  associated 
competition  for  limited  developmental  test  resources.  Given 
the  delays  observed  in  FY15,  it  is  unlikely  the  program 

will  achieve  the  currently  scheduled  Milestone  C  date  of 
April  30,  2016. 

•  Update  5  combines  an  OFP  upgrade  providing  software-driven 
radar  enhancements,  Ground  Collision  Avoidance  System 
software,  and  the  incorporation  of  limited  AIM-9X  capabilities 
The  Air  Force  Air  Combat  Command  began  a  Force 
Development  Evaluation  (FDE)  of  Update  5  OFP  software 
suite  and  limited  AIM-9X  integration.  FDE  testing  completed 
in  1QFY16. 

•  Due  to  the  Joint  Strike  Fighter  (JSF)  Program  Office’s  recent 
decision  to  terminate  funding  for  the  F-35  version  of  the 
Air  Combat  Simulator  (ACS),  there  is  a  high  likelihood  that 
a  significant  cost  transfer  to  the  F-22  program  may  occur 

in  order  to  enable  the  overall  development  of  the  ACS  and 
complete  the  integration  of  certain  advanced  threats  into  the 
ACS  battlespace  environment  necessary  to  support  Increment 
3.2B  IOT&E. 


System 

•  The  F-22A  is  an  air-superiority  fighter  that  combines  low 
observability  to  threat  radars,  sustained  high  speed,  and 
integrated  avionics  sensors. 

•  Low  observability  reduces  threat  capability  to  engage  F-22As 
with  current  adversary  weapons. 

•  The  aircraft  maintains  supersonic  speeds  without  the  use  of  an 
afterburner. 

•  Avionics  that  fuse  information  from  the  Active  Electronically 
Scanned  Array  radar,  other  sensors,  and  data  linked 
information  for  the  pilot  enable  employment  of  medium-  and 
short-range  air-to-air  missiles,  guns,  and  air-to-ground 
munitions. 

•  The  Air  Force  designed  the  F-22A  to  be  more  reliable  and 
easier  to  maintain  than  legacy  fighter  aircraft. 

•  F-22A  air-to-air  weapons  are  the  AIM-120C  radar-guided 
missile,  the  AIM-9M  infrared-guided  missile,  and  the  M61A1 
20  mm  gun. 

•  F-22A  air-to-ground  precision  strike  capability  consists  of  the 
1,000-pound  Joint  Direct  Attack  Munition  and  the  250-pound 
Small  Diameter  Bomb  Increment  1. 

•  The  F-22A  program  delivers  capability  in  increments. 
Incremental  Enhanced  Global  Strike  modernization  efforts 
include  the  following  current  and  near-term  modernization 
efforts: 

-  Increment  3.1  provides  enhanced  air-to-ground  mission 
capability,  to  include  geolocation  of  selected  emitters, 
electronic  attack,  air-to-ground  synthetic  aperture  radar 
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mapping  and  designation  of  surface  targets,  and  Small 
Diameter  Bomb  integration.  Increment  3.1  is  currently 
fielded  in  operational  F-22A  units. 

Increment  3.2A  is  a  software-only  upgrade  intended 
to  provide  improved  electronic  protection,  Link  16 
Receive,  and  combat  identification  capabilities  in  FY15. 
increment  3.2A  is  a  modernization  effort  within  the 
scope  of  the  F-22  A  Advanced  Tactical  Fighter  baseline 
acquisition  program  of  record,  increment  3.2A  is  currently 
fielding  in  operational  F-22A  units. 

Update  5  combines  an  OFP  upgrade  providing 
software-driven  radar  enhancements,  Ground  Collision 
Avoidance  System  software,  and  the  incorporation  of 
limited  AIM-9X  capabilities.  Update  5  OFP  FDE  testing 
began  in  late  FY15  and  completed  in  1QFY16. 

Increment  3.2B  is  a  separate  Major  Defense  Acquisition 
Program  modernization  effort  intended  to  integrate 
AIM  120D  and  AIM-9X  missile  systems  and  provide 
additional  electronic  protection  enhancements  and 


improved  emitter  geolocation  capability.  The  Increment 
3.2B  IOT&E  is  currently  planned  for  FY17. 

Mission 

Commanders  will  use  units  equipped  with  the  F-22A  to: 

•  Provide  air  superiority  over  friendly  and  non-permissive, 
contested  enemy  territory 

•  Defend  friendly  forces  against  fighter,  bomber,  or  cruise 
missile  attack 

•  Escort  friendly  air  forces  into  enemy  territory 

•  Provide  air-to-ground  capability  for  counter-air,  strategic 
attack,  counter-land,  and  enemy  air  defense  suppression 
missions 

Major  Contractor 

Lockheed  Martin  Aeronautics  Company  -  Fort  Worth,  Texas 


Activity 

•  The  Air  Force  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan,  FOT&E, 
and  FDE  test  plans. 

•  Increment  3.2A  developmental  testing  completed  in  early 
FY15;  however,  several  software  stability,  radar,  and  Link  16 
datalink  performance  shortfalls  remained  unresolved  during 
developmental  testing  and  were  carried  forward  into  FY15 
FOT&E. 

•  AFOTEC  conducted  FOT&E  of  Increment  3. 2 A  from 
November  2014,  through  May  2015.  During  the  FOT&E, 
a  software  anomaly  affecting  legacy  system  performance 
deficiency  was  discovered  resulting  in  a  test  pause  and 
necessitating  additional,  unplanned  developmental  testing 
efforts  to  resolve  the  associated  radar  operation  shortfall. 
Following  completion  of  the  FOT&E,  the  Air  Force  Air 
Combat  Command  released  the  Increment  3.2A  software  OFP 
for  installation  in  operational  F-22A  units  beginning  in  July 
2015. 

•  Increment  3.2B  developmental  testing  continued  throughout 
FY15,  but  experienced  delays  due  to  the  additional  regression 
testing  for  Increment  3. 2 A  and  Update  5  OFP  software  suite. 
The  Air  Force  is  projecting  a  March  2016  Milestone  C,  and 
IOT&E  is  projected  to  begin  in  spring  2017. 

•  The  JSF  Program  Office  has  recently  announced  that  it  plans 
to  cancel  the  development  of  the  Lockheed-Martin  F-35 
simulation  and  transfer  the  project  to  a  government  team. 

Both  F-22  and  F-35  JSF  simulations  currently  share  a  common 
facility  and  software  environment,  including  the  battlespace 
environment  and  threat  models.  Each  program  leverages 
new  threats  and  other  capabilities  added  by  the  other.  The 
ACS  is  a  Lockheed-Martin  facility  that  provides  the  F-22 
operational  test  community  the  ability  to  simulate  dense  threat 


environments  that  cannot  be  replicated  in  open  air.  It  has 
been  used  successfully  for  IOT&E  and  FOT&E  since  2004. 
The  ACS  includes  man-in-the-loop  and  OFP  in  the-loop  F-22 
systems,  and  capable  man-in-the-loop  adversary  systems. 

-  The  F-22  Program  Office  plans  to  use  the  ACS  for 
Increment  3.2B  for  operational  effectiveness  trials  during 
FY 17  IOT&E. 

-  Several  ACS  upgrades  are  required  to  implement  new  3.2B 
capabilities  and  to  develop  and  integrate  new  weapons  and 
threats  into  the  ACS  battlespace  environment  to  support 
operational  testing. 

-  Increment  3.2B  also  requires  the  integration  of  new 
weapons  on  the  F-22  simulation.  The  AIM-120D  model 
has  not  yet  been  delivered  to  ACS  by  the  weapon  vendor. 

•  Air  Force  Air  Combat  Command  began  an  FDE  of  the  Update 
5  OFP  software  suite.  The  FDE  completed  in  1QFY16. 

Assessment 

•  Results  of  Increment  3. 2 A  FOT&E  testing  demonstrated  that 
the  OFP  suite  is  operationally  effective;  however,  several 
unresolved  radar  and  Link  16  datalink  performance  shortfalls 
carried  forward  from  developmental  testing  and  discovered 
during  FOT&E  detract  from  overall  operational  capability. 

The  Increment  3.2A  OFP  is  not  operationally  suitable  as  it 
did  not  meet  the  Air  Force  threshold  Mean  Time  Between 
Avionics  Anomaly  requirement.  Furthermore,  based  upon 
the  demonstrated  software  stability  performance  during  the 
FOT&E,  there  is  a  high  probability  that  pilots  may  not  be  able 
to  complete  combat  missions  without  having  to  reset  or  restart 
avionics  subsystems  during  flight. 
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•  Increment  3.2A  cybersecurity  testing  consisted  of  compliance 
checks  on  security  controls.  Increment  3.2A  did  not  meet 
standards  for  multiple  compliance  checks. 

•  Increment  3.2B  developmental  testing  experienced 
several  delays  due  to  additional  unplanned  regression 
testing  for  Increment  3.2A  and  Update  5  OFP  efforts,  and 
associated  competition  for  limited  test  resources.  All  F-22A 
modernization  efforts  and  increments  are  tightly  coupled,  and 
recent  years’  reduction  in  developmental  test  force  aircraft 
and  personnel  leaves  little  margin  for  unanticipated  regression 
testing  and  correction  of  critical  deficiencies  when  discovered 
in  operational  testing.  Given  these  factors  and  delays  observed 
to  date,  it  is  likely  that  the  Air  Force  Milestone  C  decision  will 
slip  to  late  FY16. 

•  Update  5  OFP  FDE  testing  to  date  included  five  AIM-9X 
missile  live  fire  trials;  a  first  for  the  F-22A  operational  test 
community.  Four  of  the  five  performed  nominally,  yet  the  fifth 
failed  to  guide  to  the  intended  target.  The  root  cause  of  the 
miss  is  under  investigation. 

•  The  primary  schedule  risk  for  completing  ACS  weapons  model 
is  the  integration  of  a  validated  AIM- 120D  model.  Delivery 
of  the  Raytheon  AIM- 120D  model  to  Lock  heed  Martin  for 
incorporation  into  the  ACS  is  behind  schedule.  Other  weapons 
and  threats  appear  to  be  on  track  for  planned  delivery  to 
support  Increment  3.2B  IOT&E. 

•  Due  to  the  JSF  Program  Office's  recent  decision  to  terminate 
funding  for  the  F-35  version  of  the  ACS,  there  is  a  high 
likelihood  that  a  significant  cost  transfer  to  the  F-22  program 
may  occur  in  order  to  enable  the  overall  development  of  the 


ACS  and  complete  the  integration  of  certain  advanced  threats 
into  the  ACS  battlespace  environment  necessary  to  support 
Increment  3.2B  IOT&E.  Fully-funded  ACS  capabilities  are 
required  to  support  F-22  Increment  3.2B  IOT&E  adequacy, 
regardless  of  whether  or  not  F-35  funding  is  available. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
continues  to  address  previous  recommendations,  but  did  not 
resolve  Increment  3.2A  software  anomalies  and  performance 
shortfalls  before  proceeding  to  formal  AFOTEC  FY15 
FOT&E. 

•  FY 15  Recommendations.  The  Air  Force  should: 

1 .  Place  increased  emphasis  and  devote  necessary  resources 
to  ensure  system  performance  shortfalls  identified  in 
developmental  testing  are  effectively  resolved  before 
proceeding  with  operational  testing. 

2.  Improve  F-22A  avionics  software  stability  to  support 
operational  mission  execution  and  meet  Air  Force  software 
stability  requirements. 

3.  Ensure  the  adequacy  of  the  force  structure  and  schedule 
margins  necessary  to  support  forthcoming  F-22A 
developmental  testing  efforts 

4.  Ensure  adequate  funding  of  ACS  capabilities  for  F-22 
operational  test  and  evaluation  regardless  of  F-35  funding 
decisions. 

5.  Ensure  AIM-120D  models  are  delivered  and  incorporated 
into  the  ACS  to  meet  Increment  3.2B  scheduled  FY17 
IOT&E. 
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Family  of  Advanced  Beyond  Line-of-Sight  Terminals 

(FAB-T) 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  conducted  an  Operational  Assessment  (OA)  of  the 
Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 
Command  Post  Terminals  (CPTs)  from  December  5,  2014, 
through  May  30,  2015.  AFOTEC  based  the  OA  on 
Raytheon  in-plant  developmental  testing  and  government 
developmental  flight  testing  led  by  the  Air  Force’s  46th  Test 
Squadron.  Testing  was  conducted  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

•  During  the  OA,  the  FAB-T  CPT  demonstrated  nuclear 
and  non-nuclear  command  and  control  from  ground-fixed, 
ground-mobile,  and  airborne  terminals.  The  FAB-T  CPT 
also  demonstrated  the  ability  to  set  lip  and  control  networks 
and  point-to-point  communication  services  over  Milstar  and 
Advanced  Extremely  High  Frequency  (AEHF)  satellites. 

•  The  FAB-T  CPTs  experienced  hardware  failures  during  the 
OA,  and  require  additional  reliability  growth  to  demonstrate 
reaching  the  user  requirements  with  80-percent  confidence. 
Analysis  of  the  reliability  growth  tests  shows  a  large  degree 
of  uncertainty  in  the  Mean  Time  Between  Critical  Failure 
because  of  a  sparsity  of  data. 

•  In  February  2015,  USD(AT&L)  directed  the  Air  Force  to 
take  responsibility  for  the  acquisition  management  of  the 
Presidential  National  Voice  Conferencing  (PNVC)  capability 
from  the  Defense  Information  Systems  Agency  (DISA)  to 
ensure  integration  and  testing  efficiency.  The  Air  Force 
made  the  PNVC  capability  a  separate  program  under  the 
responsibility  of  the  FAB-T  program  manager.  DOT&E 
placed  PNVC  under  its  operational  test  and  evaluation 
oversight  in  July  2015. 

•  USD(AT&L)  conducted  a  Milestone  C  decision  review  for 
the  Defense  Acquisition  Board  on  September  1,  2015,  and 
authorized  the  FAB-T  program  to  enter  into  low-rate  initial 
production  for  the  first  10  terminals. 

System 

•  FAB-T  consists  of  ground  and  aircraft  communication 
terminals  with  two  terminal  types — CPTs  and  Force  Element 
Terminals  (FETs).  FAB-T  is  part  of  the  terminal  and  control 
segments  of  the  AEHF  satellite  system  and  is  designed  to 
operate  with  AEHF  Low  Data  Rate  (75  -  2,400  bits  per  second 
(bps))  and  Extended  Data  Rate  (up  to  8.192  Megabits  per 
second)  waveforms. 

•  The  CPT  is  intended  to  replace  existing  airborne  (E-4B 
and  E-6B),  ground-fixed,  and  ground-transportable  Milstar 
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command  post  terminals.  The  CPT  will  include  satellite  and 
network  control  functions,  end-user  telecommunication  device 
interfaces,  and  the  ability  to  operate  the  terminal  from  a  distant 
location  using  a  remote  node. 

•  The  FET  is  intended  to  be  installed  in  airborne  force  elements 
(B-2,  B-52,  and  RC-135).  The  FET  is  a  program  requirement 
but  is  currently  neither  funded  nor  on  contract  for  development 
and  production. 

Mission 

•  The  President,  the  Secretary  of  Defense,  Combatant 
Commanders,  and  supporting  Air  Force  component  forces 
will  use  FAB-T  to  provide  strategic  nuclear  and  non-nuclear 
command  and  control  with  EHF,  wideband,  protected,  and 
survivable  communications  terminals  for  beyond  line-of-sight 
communications. 

•  U.S.  Strategic  Command  will  use  the  FAB-T  to  perform 
satellite  Telemetry,  Tracking,  and  Commanding  functions 
for  the  AEHF/constellation,  including  management  of  the 
satellites,  communication  networks,  and  cryptologic  keys. 

Major  Contractor 

Raytheon  Net-Centric  Systems  -  Marlborough,  Massachusetts 
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Activity 

•  AFOTEC  conducted  an  OA  of  the  FAB-T  CPT  from 
December  5,  2014,  to  May  30,  2015.  AFOTEC  based  the  OA 
on  Raytheon  in-plant  developmental  testing  and  government 
developmental  flight  testing  led  by  the  Air  Force’s  46th  Test 
Squadron.  Testing  was  conducted  in  accordance  with  the 
DOT&E-approved  Test  and  Evaluation  Master  Plan  and  test 
plan. 

•  DOT&E  submitted  an  OA  report  on  the  FAB-T  CPT  on 
July  23,2015. 

•  The  contractor  conducted  reliability  growth  testing  in  the 
lab  simulating  the  operational  mission  profile  in  accordance 
with  the  FAB-T  program  manager ’s-approved  plan  from 
February  through  July  2015.  The  program  manager  tested  the 
ground-fixed  CPT  for  1,947  hours  and  the  airborne  CPT  for 
1,396  hours.  Reliability  testing  for  the  ground-transportable 
CPT  and  remote  capability  was  not  performed  during  the  OA; 
however,  the  program  manager  is  planning  to  conduct  this 
reliability  testing  after  the  OA  test  period. 

•  The  92nd  Information  Operations  Squadron  conducted  a 
Cybersecurity  Cooperative  Vulnerability  and  Penetration 
Assessment  from  December  16  -  18,  2014,  on  the 
ground-fixed  CPT  at  the  Raytheon  contractor  facility  in 
Marlborough,  Massachusetts. 

•  The  program  updated  the  DOT&E-approved  the  Test  and 
Evaluation  Master  Plan  on  August  21,  2015. 

•  In  February  2015,  USD(AT&L)  directed  the  Air  Force  to 
take  responsibility  for  the  acquisition  management  of  the 
PNVC  capability  from  D1SA  to  ensure  integration  and  testing 
efficiency.  Therefore,  DISA  is  now  supporting  the  Air  Force 
with  PNVC  baseband  equipment  development  and  engineering 
expertise  for  end-to-end  testing  of  strategic  conferencing 
services. 

•  In  May  2015,  the  Air  Force  made  the  PNVC  capability  a 
separate  program  under  the  responsibility  of  the  FAB-T 
program  manager.  DOT&E  placed  PNVC  under  its 
operational  test  and  evaluation  oversight  in  July  2015. 

•  USD(AT&L)  conducted  a  Milestone  C  decision  review  for 
the  Defense  Acquisition  Board  on  September  1,  2015,  and 
authorized  the  FAB-T  program  to  enter  into  low-rate  initial 
production  for  the  first  10  terminals. 

•  USD(AT&L)  tasked  the  FAB-T  Program  Office  to  work 
with  DOT&E  and  the  Deputy  Assistant  Secretary  of  Defense 
for  Developmental  Test  and  Evaluation  to  determine  the 
appropriate  amount  of  reliability  growth  testing  for  the  next 
phase  of  the  program. 

Assessment 

•  During  the  OA,  the  FAB-T  CPT  demonstrated  nuclear 
and  non-nuclear  command  and  control  from  ground-fixed, 
ground-mobile,  and  airborne  terminals.  The  FAB-T  CPT 


also  demonstrated  the  ability  to  set  up  and  control  networks 
and  point-to-point  communication  services  over  Milstar  and 
AEFIF.  The  FAB-T  CPT  did  not  have  the  satellite  Telemetry, 
Tracking,  and  Commanding  functions  available  to  test. 

•  Government  developmental  flight  testing  during  the  OA 
revealed  instances  where  nuclear  emergency  action  messages 
were  either  not  received  or  contained  corrupted  content. 
Failures  in  reception  occurred  after  the  ground-fixed 
FAB-T  CPT  transitioned  from  one  satellite  agile  beam  to 
another.  Failures  also  occurred  with  messages  created  by 
the  government-furnished  Milstar  Messaging  Application 
connected  to  the  ground-fixed  CPT.  Missed  or  inaccurate 
reproduction  of  the  original  message  can  cause  significant 
problems  in  the  command  and  control  of  nuclear  assets  during 
operations. 

•  The  FAB-T  program  manager  performed  root  cause  analysis 
and  determined  the  problems  experienced  during  flight  testing 
were  caused  by  faulty  software.  The  contractor  fixed  most  of 
the  software  problems  and  the  46th  Test  Squadron  conducted 
additional  flight  testing  on  June  24,  2015,  to  verify  the  fixes. 
The  46th  Test  Squadron  determined  most  of  the  problems  were 
fixed,  but  new  ones  were  found. 

•  The  FAB-T  CPTs  experienced  hardware  failures  during  the 
OA,  and  require  additional  reliability  growth  to  successfully 
demonstrate  reaching  the  user  requirements  with  80-percent 
confidence.  Analysis  of  the  reliability  growth  tests  shows  a 
large  degree  of  uncertainty  in  the  Mean  Time  Between  Critical 
Failure  because  of  a  sparsity  of  data. 

•  The  program  manager  used  a  non-standard  methodology  for 
reliability  growth  planning  and  analysis,  whereas  DOT&E 
OA  reliability  analysis  employed  the  Army  Materiel  Systems 
Analysis  Activity  methodology,  which  resulted  in  lower 
reliability  estimates. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  has 
addressed  all  previous  recommendations. 

•  FY 15  Recommendations.  The  Air  Force  should: 

1 .  Continue  to  use  reliability  growth  test  periods  to  surface 
more  failure  modes  and  correct  them  to  grow  reliability  and 
confidence  in  system  performance. 

2.  Update  the  reliability  growth  curves  with  results  from 
DOT&E ’s  OA  for  both  ground-fixed  and  airborne  CPTs. 
Updated  curves  would  allow  the  program  manager  to 
determine  how  much  more  reliability  testing  is  needed 
given  the  performance  demonstrated  during  the  OA. 

3.  Work  with  DOT&E  and  the  Deputy  Assistant  Secretary 
of  Defense  for  Developmental  Test  and  Evaluation  to 
determine  the  appropriate  amount  of  reliability  growth 
testing  for  the  next  phase  of  the  program. 
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Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center  is 
currently  conducting  a  second  KC-46A  operational  assessment 
(OA)  to  support  the  Milestone  C  decision.  DOT&E  expects 

it  to  end  February  2016,  after  the  completion  of  Milestone  C 
receiver  flights. 

•  Delays  in  Engineering  and  Manufacturing  Development 
(EMD)  aircraft  production  and  manufacturing  have  delayed 
program  test  milestones  to  include  the  Milestone  C  decision. 
DOT&E  estimates  April  2016  for  Milestone  C  and  May  2017 
for  the  start  of  IOT&E.  The  first  EMD  aircraft  (EMD-1), 

a  767-2C  variant,  began  daily  flight  testing  at  the  end  of 
July  2015.  First  flight  of  the  second  EMD  aircraft  (EMD  2), 
the  first  fully  configured  tanker  variant,  began  flights  in  late 
September  2015.  Aerial  refueling  certification  of  11  of  the 
18  different  receiving  aircraft  planned  for  EMD  will  not  be 
complete  until  after  the  start  of  the  IOT&E. 

•  Testing  in  the  Boeing  lighting  lab  and  wet  fuels  lab,  each 
containing  full-up  installations  of  the  respective  aircraft 
systems  is  anticipated  to  complete  in  2QFY16. 

•  The  Air  Force  successfully  completed  the  ballistic  test 
phase  of  the  live  fire  test  program.  Ballistic  testing  in  FY15 
investigated  the  KC-46A’s  response  to  dry  bay  fires  and 
structural  damage  to  the  engines  and  engine  pylon  due  to 
impact  from  man-portable  air-defense  systems  (MANPADS). 
Preliminary  review  of  the  data  did  not  reveal  any  unknown 
vulnerabilities  to  the  threats  tested. 

System 

•  The  KC-46A  aerial  refueling  aircraft  is  the  first  increment  of 
replacement  tankers  (179)  for  the  Air  Force’s  fleet  of  KC-135 
tankers  (more  than  400). 

•  The  KC-46A  design  uses  a  modified  Boeing  767-200ER 
commercial  airframe  with  numerous  military  and 
technological  upgrades,  such  as  the  fly-by-wire  refueling 
boom,  the  remote  air  refueling  operator’s  station,  787  cockpit, 
additional  fuel  tanks  in  the  body,  and  defensive  systems. 

•  The  KC-46A  will  provide  both  a  boom  and  probe-drogue 
refueling  capabilities.  The  KC-46A  is  equipped  with  an  air 
refueling  receptacle  so  that  it  can  also  receive  fuel  from  other 
tankers,  including  legacy  aircraft. 

•  The  KC-46A  is  designed  to  have  significant  palletized  cargo 
and  aeromedical  capacities;  chemical,  biological,  radiological, 
nuclear  survivability;  and  the  ability  to  host  communications 
gateway  payloads. 


•  Survivability  enhancement  features  are  incorporated  into  the 
KC-46A  design. 

-  Susceptibility  is  reduced  with  an  Aircraft  Survivability 
Equipment  suite  consisting  of  Large  Aircraft  Infrared 
Countermeasures,  the  ALR-69A  Radar  Warning  Receiver 
(RWR),  and  a  Tactical  Situational  Awareness  System.  The 
suite  is  intended  to  compile  threat  information  from  the 
ALR-69A  RWR  and  other  on-  and  off-board  sources  and  to 
prompt  the  crew  with  an  automatic  re-routing  suggestion  in 
the  event  of  an  unexpected  threat. 

-  Vulnerability  is  reduced  by  adding  fuel  tank  inerting  and 
integral  armor  to  provide  some  protection  to  the  crew  and 
critical  systems. 

Mission 

Commanders  will  use  units  equipped  with  the  KC-46A  to: 

•  Perform  air  refueling  to  accomplish  six  primary  missions 
to  include  nuclear  operations  support,  global  strike  support, 
air  bridge  support,  aircraft  deployment,  theater  support,  and 
special  operations  support.  Secondary  missions  will  include 
airlift,  aeromedical  evacuation,  emergency  aerial  refueling,  air 
sampling,  and  support  of  combat  search  and  rescue. 

•  Operate  in  day/night  and  adverse  weather  conditions  globally 
to  support  U.S.,  joint,  allied,  and  coalition  forces. 

•  Operate  in  a  non-permissive  environment. 

Major  Contractor 

The  Boeing  Company,  Commercial  Aircraft  in  conjunction  with 

Defense,  Space  &  Security  -  Seattle,  Washington 
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Activity 

•  DOT&E  approved  the  post-Milestone  B  Test  and  Evaluation 
Master  Plan  (TEMP)  in  January  2013,  with  concerns  about 
adequate  calendar  time  for  correction  of  discrepancies  or 
deficiencies  between  the  end  of  developmental  testing  and  the 
beginning  of  IOT&E.  DOT&E  has  taken  this  into  account 
while  working  with  the  program  to  develop  the  Milestone  C 
TEMP. 

•  DOT&E  approved  the  Air  Force  Operational  Test  and 
Evaluation  Center’s  second  KC-46AOA-2  plan  in  May  2015 
to  support  the  Milestone  C  decision.  This  plan  included 
revised  survey  methodology  consistent  with  DOT&E’s 
guidance.  Delays  in  EMD  aircraft  production  and 
manufacturing  have  led  to  two  extensions  in  the  completion 
date  for  OA-2;  DOT&E  expects  OA-2  to  end  February  2016, 
after  the  air  refueling  demonstration  flights  in  support  of 
Milestone  C. 

•  Developmental  and  Federal  Aviation  Administration  (FA A) 
test  planning  is  substantially  complete.  The  Air  Force 
accepted  the  contractor’s  Stage  4  (final  build)  test  plans  in 
January  2015.  Some  test  plans  have  been  revised  throughout 
the  year  due  to  FAA  test  planning  or  to  improve  test  schedule 
efficiency. 

•  First  flight  of  the  first  EMD  aircraft  occurred 

December  28,  2014,  6  months  late,  primarily  due  to  electrical 
wiring  design  problems.  After  a  single  flight,  the  aircraft 
was  down  for  planned  finishing  work;  this  down  period  was 
extended  by  a  necessary  redesign  of  the  fuel  pump  manifold. 
The  aircraft  began  near  daily  flight  testing  at  the  end  of 
July  2015. 

•  EMD  2,  the  first  fully  configured  tanker  variant,  began  flying 
in  late  September.  The  planned  first  flight  date  was  January 
2015  as  documented  in  the  post-Milestone  B  TEMP. 

•  Testing  in  the  lighting  lab  and  wet  fuels  lab,  each  containing 
full-up  installations  of  the  respective  aircraft  systems,  is 
anticipated  to  complete  in  2QFY16. 

•  The  Air  Force  used  a  standard  Air  Force  MJ-1  weapons 
loader  to  install  a  boom  on  a  KC-46  to  show  the  feasibility 
of  using  current  Aerospace  Ground  Equipment  rather  than 
developing  new,  unique  equipment.  The  Air  Force  completed 
a  refueling  surge  pressure  test  on  a  KC-135  and  on  a  KC-10 
to  develop  baseline  data  for  comparison  to  surge  pressures 

in  the  KC-46A.  The  technical  order  verification  process  is 
approximately  one  quarter  complete  and  should  be  finished 
prior  to  IOT&E. 

•  All  parties  have  agreed  upon  the  open-air  test  venues  for  the 
ALR-69A  RWR  and  the  AAQ-24  Large  Aircraft  Infrared 
Countermeasures  (LAIRCM)  system  and  detailed  test 
planning  for  each  venue  is  near  completion. 

•  Testers  have  completed  laboratory  tests  of  the  defensive 
systems  including  the  ALR-69A  RWR  and  the  AAQ  24 
LAIRCM.  Flight  test  planning  for  LAIRCM  is  nearly 
complete,  while  flight  test  planning  for  the  RWR  remains  to  be 
completed.  Both  flight  tests  are  planned  for  fall  2016. 

•  The  program  conducted  three  live  fire  test  series  completing 
the  ballistic  test  portion  of  the  LFT&E  program.  The 


Air  Force  conducted  testing  in  accordance  with  the 
DOT&E-approved  LFT&E  strategy. 

•  The  Air  Force  is  nearing  completion  of  the  KC-46A 
survivability  assessment  against  radar  guided  surface-to-air 
missiles  and  the  performance  of  the  ALR-69A  RWR  against 
selected  threats.  Survivability  evaluation  against  other 
operationally  relevant  threats,  crew  casualty  assessment,  and 
non-kinetic  threat  vulnerability  analyses  also  remain  to  be 
completed. 

•  Boeing  prepared  a  test  plan  for  Electromagnetic  Pulse  (EMP) 
testing  based  on  the  contract  specified  design  margin  of 

6  decibels  (dB). 

•  The  program  accomplished  an  initial  cybersecurity 
Cooperative  Vulnerability  Penetration  Assessment  (CVPA) 
in  one  of  the  Boeing  system  integration  labs  in  August  2015. 
Future  cybersecurity  test  plans  include  additional  CVPAs 
accomplished  in  the  system  integration  labs  and  on  the 
aircraft,  followed  by  an  Adversarial  Assessment  accomplished 
on  the  aircraft  during  the  IOT&E. 

Assessment 

•  DOT&E  assessed  the  ALR-69A  RWR  as  installed  on  the 
C-130F1  as  not  operationally  effective  but  operationally 
suitable  based  on  tests  conducted  by  the  Air  Force  in 
October  2012. 

-  The  system  did  not  consistently  provide  the  aircrew 
timely  and  accurate  threat  information  and  the  system 
demonstrated  a  random  threat  symbol  splitting  deficiency. 
Threat  symbol  splitting  occurs  when  one  threat  signal 
received  by  the  system  produces  multiple  threat  symbols 
at  different  azimuths  on  the  cockpit  display.  This  degrades 
the  aircrew’s  situational  awareness  as  to  which  displayed 
threats  are  “real”  and  where  those  real  threats  are  located, 
and  inhibits  the  aircrew’s  ability  to  appropriately  react  to 
the  threat(s)  in  a  timely  manner.  The  details  are  presented 
in  DOT&E’s  classified  IOT&E  report  dated  October  2012. 

-  Although  the  Air  Force  System  Program  Office  and 
Raytheon  conducted  hardware-in-the-loop  (HW1L)  tests  to 
demonstrate  the  threat  signal  splitting  deficiency  has  been 
resolved,  DOT&E  does  not  think  HW1L  testing  by  itself 

is  adequate  to  verify  the  deficiency  has  been  resolved  and 
that  the  software  update  did  not  induce  any  other  adverse 
system  performance. 

•  The  delay  in  first  flight  of  the  KC-46A  has  altered  the  planned 
certification  schedule  of  air  refueling  receiving  aircraft  and 
accomplishment  of  the  26  Milestone  B  Acquisition  Decision 
Memorandum  Technical  Performance  Measures  entrance 
criteria.  This,  in  turn,  will  delay  the  August  2015  Milestone  C 
decision  (based  on  the  post-Milestone  B  TEMP  schedule)  until 
April  2016. 

•  DOT&E  identified  several  shortfalls  in  the  planned  test 
program  that  require  resolution  prior  to  Milestone  C  TEMP 
approval. 

-  A  current  schedule  based  on  reasonable  test  efficiencies 
must  be  included. 
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-  Planning  should  allow  the  operational  test  aircrew  and 
maintenance  personnel  adequate  time  to  develop  system 
proficiency  to  support  IOT&E. 

-  Sufficient  calendar  time  must  be  allotted  for  correction 
of  discrepancies  and/or  deficiencies  discovered  during 
developmental  testing  prior  to  the  planned  start  of 
operational  testing. 

-  Cyber  vulnerability  and  penetration  testing  to  date  has 
not  tested  vulnerabilities  on  a  production  representative 
aircraft.  Tentatively  scheduled  events  are  expected  to 
fulfill  this  requirement,  but  need  to  be  formally  coordinated 
and  scheduled. 

•  The  expected  IOT&E  start  date  has  been  delayed  10  months 
from  the  post-milestone  B  TEMP  schedule.  DOT&E  analysis 
of  Boeing  progress  and  schedules  with  regard  to  aerial 
refueling  certifications  and  operator/maintainer  training 
indicates  that  operational  testing  will  likely  be  delayed  a 
total  of  12  months.  This  is  in  line  with  previous  DOT&E 
estimates.  Planning  since  Milestone  B  has  moved  air  refueling 
certification  of  11  of  the  18  different  receiving  aircraft  until 
after  the  start  of  IOT&E. 

•  Testing  completed  to  date  and  planned  testing  of  LAIRCM 
should  be  adequate  to  determine  system  effectiveness 

and  suitability.  Analyses  of  LAIRCM  analytic  model  and 
laboratory  test  results  are  ongoing.  To  date,  no  significant 
discrepancies  from  expected  performance  have  been  noted. 

•  Flight  testing  of  the  RWR  now  includes  testing  at  a 
government  electronic  warfare  test  range  in  addition  to  a 
training  range.  Initial  planning  for  this  test  has  begun. 

•  Preliminary  analyses  of  live  fire  test  data: 

-  Confirmed  the  vulnerability  of  the  KC-46A  to 
threat-induced  dry  bay  fires,  including  the  wing-leading 
edge,  wing-trailing  edge,  center  wing  dry  bay,  and 
fuselage  body  tank  dry  bays.  The  Air  Force  assessed  the 
vulnerability  to  several  threats  including  small  arms  and 
missile  fragments.  All  live  fire  ballistic  test  results  will 
be  incorporated  into  the  vulnerability  analysis  to  confirm 
whether  the  KC-46A  meets  the  30-minute  controlled  flight 
vulnerability  specification  in  FY16. 

-  Quantified  the  threat-induced  structural  limitations  of  the 
KC-46A  wings  for  selected  engagement  conditions.  The 
Air  Force  will  assess  the  survivability  of  the  wing  structure 
to  a  range  of  small  arms,  anti-aircraft  artillery  threats,  and 
MANPADS  engagements. 

-  Demonstrated  the  vulnerability  of  the  engine  and  the 
engine  pylon  to  two  specific  MANPADS  engagement 
conditions.  F  urther  analyses  will  assess  the  resultant 
aircraft  survivability  for  engagement  conditions,  not  tested. 

-  Demonstrated  the  expected  cockpit  and  boom  operator 
station  armor  effectiveness  against  the  specification  threat 
with  80  percent  confidence  and  assessed  the  effectiveness 
of  the  installed  armor  against  the  specification  threat 

and  two  other  operationally  representative  threats.  The 
program  will  complete  an  evaluation  of  the  effects  of 
these  data  on  the  overall  crew  protection  assessment  in 
FY16.  Live  Fire  ballistic  test  results  incorporated  into  the 


vulnerability  analysis  confirms  the  KC-46A  is  meeting  the 
crew  station  armor  vulnerability  specification  against  the 
specification  threat. 

-  Supported  the  updates  to  the  initial  aircraft  vulnerability 
assessment,  which  quantified  the  aircraft’s  vulnerabilities 
to  the  specification  and  other  expected  threats  for  a  range 
of  operationally  relevant  engagement  conditions,  not 
tested.  Boeing  is  scheduled  to  deliver  their  final  analysis  to 
the  Air  Force  in  FY16. 

•  The  KC-46A  EMP  design  margin  was  based  on  Military 
Standard  (MIL  STD)-464  and  the  threat  defined  in 
MIL-STD-2169.  After  the  fixed-price  contract  was  awarded, 
the  DOD  instituted  a  new  MIL  STD-3023  that  requires 
tanker  aircraft  to  meet  a  20  dB  EMP  design  margin  versus 
the  contractually  required  6  dB  EMP  design  margin.  Unless 
additional  tests  are  resourced,  the  Air  Force  or  the  U.S. 
Strategic  Command  will  not  know  if  the  KC-46A  meets  the 
20  dB  EMP  hardening  requirement  in  MIL-STD-3023. 

•  During  the  CVPA,  testers  discovered  several  vulnerabilities. 
The  program  plans  to  correct  some  of  them  shortly  while 
corrections  to  others  that  are  related  to  government  furnished 
equipment  are  under  discussion. 

•  The  integrated  test  team  is  working  a  cybersecurity  strategy 
consistent  with  DOT&E  guidance;  however,  specific  details  to 
conduct  an  adequate  operational  test  are  not  yet  defined. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
addressed  two  of  the  FY12  recommendations  to  incorporate 
realistic  assumptions  in  test  plans  and  provide  a  plan  for  air 
refueling  receiver  certification;  however,  additional  work  is 
still  needed.  The  Air  Force  still  needs  to  address  the  remaining 
FY12  and  FY13  recommendations  to: 

1 .  Submit  a  TEMP  with  a  schedule  mitigating  the  shortfalls 
that  may  adversely  affect  IOT&E. 

2.  Provide  an  approach  to  correct  the  ALR-69A  RWR 
shortfalls  prior  to  integration  on  the  KC-46A. 

3.  Plan  to  begin  IOT&E  at  least  12  months  later  than  the 
post-milestone  B  TEMP  indicates  to  allow  for  completion 
of  developmental  test  and  initial  training. 

4.  Formally  plan  testing  against  realistic  cybersecurity  threats 
conducted  on  a  production  representative  aircraft  to  identify 
vulnerabilities  for  correction.  In  addition,  plan  follow-on 
penetration  testing  to  assess  performance  in  terms  of 
protect,  detect,  react,  and  restore  functions. 

•  FY 15  Recommendations.  The  Air  Force  should: 

1.  Ensure  all  air  refueling  receiver  aircraft  are  certified  for  use 
by  operational  aircrew  early  enough  in  IOT&E  to  permit 
sufficient  operational  testing. 

2.  In  conjunction  with  U.S.  Strategic  Command,  determine 
whether  its  personnel  can  conduct  the  nuclear  deterrence 
and  strike  missions  with  a  KC-46A  only  having  6  dB  EMP 
shielding  as  per  the  contract.  If  additional  EMP  shielding 
is  deemed  necessary,  the  Air  Force  should  conduct  testing 
as  part  of  FOT&E  to  determine  the  actual  KC-46A  EMP 
design  margin. 
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Massive  Ordnance  Penetrator  (MOP) 


Executive  Summary 

•  In  February  2015,  the  Air  Force  successfully  completed  one 
weapon  drop  from  the  B-2  aircraft  on  a  representative  target, 
and  in  April  2015,  completed  one  weapon  drop  from  the  B-2 
aircraft  on  a  concrete  slab.  These  tests,  conducted  at  the  White 
Sands  Missile  Range  (WSMR),  New  Mexico,  demonstrated 
weapon  effectiveness  after  the  Air  Force  incorporated  planned 
enhancements  and  completed  the  Enhanced  Threat  Reduction 
(ETR)  Phase  2  testing. 

•  DOT&E  published  a  classified  Early  Fielding  Report 
summarizing  the  ETR  Phase  2  test  results  in  April  2015. 

System 

•  The  GBU-57  Massive  Ordnance  Penetrator  (MOP)  is  a 
large,  GPS-guided,  penetrating  weapon  with  the  ability  to 
attack  deeply-buried  and  hardened  bunkers  and  tunnels.  The 
warhead  case  is  made  from  a  special  high-performance  steel 
alloy  and  its  design  allows  for  a  large  explosive  payload  while 
maintaining  the  integrity  of  the  penetrator  case  during  impact. 

•  The  B-2  Spirit  is  the  only  aircraft  in  the  Air  Force  programmed 
to  employ  the  MOP. 

•  The  GBU-57  warhead  is  more  powerful  than  its  predecessors, 
the  BLU-109  and  GBU-28. 

•  The  MOP  is  an  Air  Force-led,  Quick  Reaction  Capability  that 
is  a  Secretary  of  Defense  special  interest  effort  and  is  under 
DOT&E  oversight. 


Mission 

Combatant  Commanders  use  MOP  to  conduct  pre-planned,  day 
or  night  attacks  against  defended  point  targets  vulnerable  to  blast 
and  fragmentation  effects  and  requiring  significant  penetration, 
such  as  hardened  and  deeply-buried  facilities. 

Major  Contractor 

The  Boeing  Company,  Defense,  Space  & 

Security  -  St.  Louis,  Missouri 


Activity 

•  In  January/February  2015,  the  Air  Force  conducted  one  live 
weapon  drop  at  WSMR,  on  a  representative  target  to  evaluate 
weapon  effectiveness.  An  Air  Force  B-2  aircraft  flew  two 
missions  to  complete  the  drop;  telemetry  problems  prevented 
weapon  release  on  the  first  mission. 

•  In  April  2015,  the  Air  Force  conducted  an  inert  weapon  drop  at 
WSMR  on  a  concrete  slab  target.  This  testing  was  to  evaluate 
the  effect  of  the  ETR  Phase  2  modifications  to  the  weapon. 

An  Air  Force  B-2  aircraft  flew  one  mission  to  complete  the 
drop. 

•  Both  flight  tests  were  successful  and  completed  the  ETR 
Phase  2  test. 

•  DOT&E  submitted  a  classified  Early  Fielding  Report  in 
April  2015  detailing  the  results  of  ETR  Phase  2. 


Assessment 

•  A  problem  with  telemetry  data  prevented  weapon  release  in 
the  planned  January  2015  test.  The  program  successfully 
completed  the  test  in  February  2015. 

•  Both  of  the  flight  tests  were  successful  and  demonstrated 
weapon  effectiveness  with  ETR  Phase  2  modifications. 

•  The  Air  Force  will  continue  with  ETR  Phase  3  testing  in 
FY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  There  were  no  previous 
recommendations  for  this  program. 

•  FY15  Recommendations.  None. 
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Miniature  Air-Launched  Decoy  (MALD)  and 
MALD- Jammer  (MALD-J) 


Executive  Summary 

•  Miniature  Air  Launched  Decoy  -  Jammer  (MALD-J)  is 
operationally  effective  with  limitations  and  operationally 
suitable  in  specific  environments  as  detailed  in  DOT&E’s 
March  2015  classified  IOT&E  report.  Not  all  operational 
environments  could  be  assessed.  The  Air  Force  Operational 
Test  and  Evaluation  Center  completed  mission  level 
simulation  testing,  but  certain  operational  elements  were  not 
included  because  of  limitations  in  the  Digital  Integrated  Air 
Defense  System  (DIADS). 

•  The  Program  Office  completed  open  air  testing  for  MALD-J 
(and  MALD)  with  a  software  upgrade  to  address  problems 
identified  in  GPS-degraded  or  denied  environments. 

•  Suitability  data  show  that  the  Air  Force’s  corrective  actions 
have  improved  the  materiel  reliability  for  MALD-J  (and 
MALD). 

System 

•  MALD  is  a  small,  low-cost,  expendable,  air-launched  vehicle 
that  replicates  how  fighter,  attack,  and  bomber  aircraft  appear 
to  enemy  radar  operators. 

•  MALD-J  is  an  airborne  stand-in  jammer  for  electronic  attack 
with  the  ability  to  loiter  on  station. 

•  MALD-J  will  jam  specific  Early  Warning/Ground  Control 
Intercept/ Acquisition  radars  while  retaining  the  capabilities  of 
the  MALD. 

•  MALD-J  will  stimulate  and  degrade  an  enemy’s  integrated  air 
defense  system. 

•  The  F-16  C/D  and  B-52H  are  the  lead  aircraft  to  employ 
MALD  and  MALD-J. 


Mission 

•  Combatant  Commanders  will  use  MALD  and  MALD-J  to 
improve  battlespace  access  for  airborne  strike  forces  by 
deceiving,  distracting,  or  saturating  enemy  radar  operators  and 
Integrated  Air  Defense  Systems. 

•  Combatant  Commanders  will  use  the  MALD  to  allow  an 
airborne  strike  force  to  accomplish  its  mission  by  deceiving 
enemy  radars  and  air  defense  systems  to  treat  MALD  as  a 
viable  target. 

•  Combatant  Commanders  will  use  the  MALD-J  to  allow  an 
airborne  strike  force  to  accomplish  its  mission  by  jamming 
specific  enemy  radars  and  air  defense  systems  to  degrade  or 
deny  detection  of  friendly  aircraft  or  munitions. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  In  October  2013,  the  Air  Force  Operational  Test  and 
Evaluation  Center  completed  full  mission-level  simulation 
testing  using  the  DIADS. 

•  The  28th  Test  and  Evaluation  Squadron  conducted  a  Force 
Development  Evaluation  (FDE),  in  conjunction  with  a 
MALD-J  Reliability  Assessment  Program  mission,  to 
address  problems  identified  in  FY14.  DOT&E  observed  this 
evaluation  and  used  the  FDE  as  an  operational  test. 

•  On  October  24,  2015,  during  U.S.  Air  Forces  Central 
Command  characterization  testing,  MALD-J  aborted  its 
mission  because  of  large  values  in  the  Geometric  Dilution  of 
Precision  (GDOP)  of  GPS  satellites  used  in  navigation.  The 
navigation  system  did  not  update  the  satellites  to  be  used  in 
the  calculation  of  GDOP  prior  to  launch. 


•  The  FDE  test  included  the  launch  of  six  MALD-J  vehicles  and 
one  MALD  vehicle  with  the  software  upgrade,  which  includes 
barometric  aiding  in  the  GPS  Aided  Inertial  Navigation 
System  (GAINS). 

•  In  March  2015,  DOT&E  submitted  an  IOT&E  report. 

•  In  March  2015,  the  Program  Office  completed  the  Critical 
Design  Review  for  the  obsolescence  upgrade  to  the  GAINS, 
designating  it  GAINS2. 

•  All  MALD-J  (and  MALD)  testing  was  conducted  in 
accordance  with  a  DOT&E  approved  test  plan. 

Assessment 

•  The  March  2015  IOT&E  report  concluded  MALD-J  is 
operationally  effective  with  limitations  and  operationally 
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suitable  in  specific  environments.  Not  all  operational 
environments  could  be  assessed  because  of  simulation 
limitations  ofDlADS. 

•  The  problem  of  the  MALD-J  aborting  due  to  the  navigation 
system  not  updating  satellites  in  the  GDOP  calculation  prior  to 
launch  will  be  corrected  in  software  build  nine. 

•  Barometric  aiding  improved  vertical  navigational  accuracy 
(and  stability)  in  a  GPS-degraded  or  denied  environment. 

No  improvement  in  horizontal  navigational  accuracy  was 
observed. 

•  A  combination  of  MALD  and  MALD-J  data  were  used  to 
show  a  materiel  reliability  of  86±2  percent.  Corrective 
actions  have  improved  the  material  reliability  of  MALD-J 
(and  MALD).  Since  no  failures  in  the  MALD-J  payload 
have  occurred  to  date,  and  the  other  systems  are  otherwise 
essentially  identical,  combining  these  data  is  appropriate. 

•  Mission  planning  test  events  determined  B-52  aircrews 
planning  a  full  load  of  1 6  MALD-J  may  exceed  the  time 
required  to  support  a  72-hour  Air  Tasking  Order  planning 
cycle. 

-  The  mission  planning  times  from  the  IOT&E 
mission-planning  events  were:  one  MALD-J  was 
40  minutes,  which  is  well  under  the  7-hour  requirement  to 
plan  a  single  MALD-J;  a  fully-loaded  F-16  (4-MALD-Js) 


was  an  average  of  2  hours  and  15  minutes;  and  a 
fully-loaded  B-52  (16  MALD-Js)  was  an  average  of  over 
11  hours.  The  time  to  mission  plan  a  full  load  B-52  is 
assessed  as  excessive  and  potentially  unable  to  support  a 
72-hour  Air  Tasking  Order  planning  cycle. 

•  In  FY16,  the  Air  Force  intends  to  conduct  mission  planning 
events  to  demonstrate  improved  mission  planning  time 
performance. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
satisfactorily  addressed  one  of  two  FY14  recommendations. 
However,  they  still  need  to  improve  mission-planning 
capabilities  for  the  MALD-J  program  to  reduce  the  time 
needed  to  plan  a  full  load  of  MALD-J  vehicles. 

•  FY 15  Recommendations.  The  Air  Force  should: 

1 .  Incorporate  additional  operational  elements  into  the 
mission-level  simulation  in  D1ADS. 

2.  Improve  horizontal  navigational  accuracy  of  the  MALD-J 
(and  MALD)  vehicle. 

3.  Test  the  GAINS2  obsolescence  upgrade  to  confirm 
equivalent  or  improved  performance  compared  to  the 
GAINS. 
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MQ-9  Reaper  Armed  Unmanned  Aircraft  System  (UAS) 


Executive  Summary 

•  The  MQ-9  program  continues  to  face  systemic  challenges 
in  prioritizing  development  and  testing  efforts  between 
evolving  program  of  record  requirements  and  other  capability 
development  efforts  outside  of  the  system’s  program  of  record. 
Evolving  priorities  and  competing  development  and  fielding 
desires  continue  to  stress  operational  test  agency  capacity 

to  support  both  program  of  record  testing  requirements  and 
accelerated  fielding  of  capabilities  desired  by  the  Air  Force. 

•  Planned  FY14  FOT&E  of  the  Block  5  Remotely  Piloted 
Aircraft  (RPA),  Block  30  Ground  Control  Station  (GCS), 
and  software  Operational  Flight  Program  (OFP)  904.6  did 
not  occur  due  to  additional  developmental  testing  to  address 
thermal  and  power  management  problems.  FOT&E  did  not 
occur  in  FY15.  A  combination  of  additional  Block  5  RPA 
developmental  testing  and  competing  non-program  of  record 
development  and  test  priorities  overtaxed  available  operational 
test  resources  needed  to  support  planned  FY15  FOT&E.  This 
FOT&E  is  planned  to  begin  in  2QFY16. 

•  General  Atomics  delivered  the  final  Block  1  RPA  to  the  Air 
Force  in  2QFY15  and  transitioned  the  production  line  to 
Block  5  RPAs.  There  is  no  Full-Rate  Production  decision 
associated  with  MQ-9  deliveries;  the  Air  Force  will  complete 
the  MQ-9  fleet  acquisition  under  low-rate  initial  production. 

•  The  current  DOT&E-approved  MQ-9  Test  and  Evaluation 
Master  Plan  (TEMP)  supports  FOT&E  of  the  Block  5  RPA, 
Block  30  GCS,  and  OFP  904.6.  Upon  completion  of  FOT&E 
of  this  configuration,  a  new  TEMP  will  be  required  to 
document  the  incorporation  of  new  program  of  record  content 
(e.g.,  the  Block  50  GCS)  into  the  MQ-9  Increment  1  system. 

System 

•  The  MQ-9  Reaper  Unmanned  Aircraft  System  (UAS)  is  a 
remotely  piloted,  armed,  air  vehicle  that  uses  optical,  infrared, 
and  radar  sensors  to  locate,  identify,  target,  and  attack  ground 
targets. 

-  The  MQ-9  RPA  is  a  medium-sized  aircraft  that  has 
an  operating  ceiling  up  to  50,000  feet,  an  internal 
sensor  payload  of  800  pounds,  an  external  payload 
of  3,000  pounds,  and  an  endurance  of  approximately 
14  hours. 

-  The  GCS  commands  the  MQ-9  RPA  for  launch,  recovery, 
and  mission  control  of  sensors  and  weapons.  C  band 
line-of-sight  datalinks  are  used  for  RPA  launch  and 
recovery  operations,  and  Ku-band  satellite  links  are  used 
for  RPA  mission  control. 


•  The  MQ-9  RPA  carries  AGM-1 14,  FIELLFIRE  II  anti-armor 
precision  laser-guided  missiles,  and  GBU-12  500  pound  laser 
guided  bombs. 

•  The  Air  Force  is  using  an  evolutionary  acquisition  approach 
for  meeting  Increment  1  Capability  Production  Document 
requirements,  with  Block  1  and  Block  5  RPAs  and  Block  15 
and  Block  30  GCSs. 

•  The  Air  Force  is  currently  fielding  the  Block  1  RPA  and  the 
Block  15  GCS. 

•  The  Air  Force  designed  the  Block  5  RPA  to  incorporate 
improved  main  landing  gear,  an  upgraded  electrical  system 
with  more  power,  an  additional  ARC-210  radio,  encrypted 
datalinks,  a  redesigned  avionics  bay  and  digital  electronic 
engine  control  system,  the  BRU-71  bomb  rack,  high-definition 
video,  and  upgraded  software  to  allow  the  two-person  aircrew 
to  operate  all  onboard  systems. 

Mission 

•  Combatant  Commanders  use  units  equipped  with  the  MQ-9  to 
conduct  armed  reconnaissance  and  pre-planned  strikes.  Units 
equipped  with  MQ-9s  can  find,  fix,  track,  target,  engage,  and 
assess  critical  emerging  targets  (both  moving  and  stationary). 

•  MQ-9  units  can  also  conduct  aerial  intelligence  gathering, 
reconnaissance,  surveillance,  and  target  acquisition  for  other 
airborne  platforms. 

Major  Contractor 

General  Atomics  Aeronautical  Systems  Inc.  -San  Diego, 

California 
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Activity 

•  The  Air  Force  conducted  all  MQ-9  testing  in  accordance  with 
the  DOT&E-approved  test  plan  and  TEMP. 

•  The  Air  Force  completed  additional  formal  developmental 
testing  on  OFP  version  904.6  Revision  K  for  the  Block  5  RPA 
and  Block  30  GCS  in  2QFY15.  This  software  revision 
incorporated  fixes  to  deficiencies  and  will  be  the  OFP 
evaluated  in  the  forthcoming  FOT&E. 

•  FOT&E  did  not  occur  in  FY15.  A  combination  of  additional 
Block  5  RPA  developmental  testing  to  address  thermal  and 
power  management  problems  and  competing  non-program 
of  record  development  and  test  priorities  overtaxed  available 
operational  test  resources  needed  to  support  planned  FY15 
FOT&E.  FOT&E  will  not  begin  until  2QFY16. 

-  The  Air  Force  purchased  a  new,  more  powerful  5-ton 
cooling  cart  and  modified  the  Block  5  RPA  with  a  plenum 
to  direct  cooled  air  into  the  forward  avionics  bay  to 
increase  cooling  capacity.  Developmental  testing  in 
4QFY15  demonstrated  that  these  measures  mitigated 
previous  FY14  transmitter  overheating  problems. 

-  Thermal  and  power  management  problems  led  to  FY15 
developmental  test  mission  ground  aborts  due  to  depleted 
batteries  that  become  too  hot  for  charging  on  the  ground 
prior  to  takeoff.  As  of  the  end  of  FY15,  the  Air  Force 
had  not  determined  a  resolution  to  ongoing  overheating 
problems. 

•  General  Atomics  delivered  the  final  Block  1  RPA  to  the  Air 
Force  in  2QFY15  and  transitioned  the  production  line  to 
Block  5  RPAs.  As  of  3QFY15,  General  Atomics  had  delivered 
199  of  364  RPAs  (Block  1  and  Block  5  combined)  to  the  Air 
Force.  General  Atomics  plans  to  deliver  the  final  Block  5  RPA 
in  FY21.  There  is  no  Full-Rate  Production  decision  associated 
with  MQ-9  deliveries;  the  Air  Force  will  complete  the  MQ-9 
fleet  acquisition  under  low-rate  initial  production. 

•  The  final  configuration  of  the  MQ-9  Increment  1  UAS 
continued  to  evolve  throughout  FY15.  The  Air  Force  intends 
to  incorporate  an  improved  Multi  Spectral  Targeting  System-B 
electro-optical/infrared  sensor,  additional  weapons,  avionics 
hardware,  and  further  system  software  revisions  into  the 
Increment  1  program  of  record  capabilities. 

•  In  FY15,  the  MQ-9  UAS  Program  Office  adopted  a  new 
hybrid  acquisition  strategy  approach  in  response  to  changing 
non-program  of  record  content  desired  by  the  Air  Force  and 
for  delivering  desired  additional  capabilities.  Candidate 
capabilities  are  intended  to  be  delivered  in  a  series  of  bundled 
software/hardware  releases  under  an  accelerated  development 
and  testing  schedule  as  an  extension  of  the  existing  program  of 
record. 

Assessment 

•  The  MQ-9  program  continues  to  face  systemic  challenges 
in  prioritizing  development  and  testing  efforts  between 
evolving  program  of  record  requirements  and  other  capability 


development  efforts  outside  of  the  system’s  program  of  record. 
Evolving  priorities  and  competing  development  and  fielding 
desires  continue  to  stress  operational  test  agency  capacity 
to  support  both  program  of  record  testing  requirements  and 
accelerated  fielding  of  capabilities  desired  by  the  Air  Force. 

•  Thermal  management  and  overheating  problems  identified  in 
FY14  were  not  fully  resolved  in  FY15  despite  the  introduction 
of  Block  5  RPA  design  changes  and  the  introduction  of 
additional  ground  cooling  equipment.  Although  these 
measures  mitigated  RPA  forward  avionics  bay  redundant 
control  module  and  transmitter  overheating  shortfalls, 

4QFY15  hot  weather  developmental  testing  revealed  power 
management  problems  that  preclude  charging  batteries  on 

the  ground  leading  to  depleted  batteries  prior  to  takeoff 
and  leading  to  mission  aborts.  After  an  operationally 
representative  taxi  time  in  warm  to  hot  weather,  batteries  that 
become  too  hot  will  be  inhibited  from  charging  to  prevent 
battery  overheating.  If  this  occurs,  required  battery  back-up 
power  for  takeoff  may  be  insufficient  and  the  mission  could 
be  aborted.  As  of  the  end  of  FY15,  the  Air  Force  had  not 
identified  a  solution  to  the  ongoing  Block  5  RPA  overheating 
problems.  A  combination  of  additional  Block  5  RPA 
developmental  testing  and  competing  non-program  of  record 
development  and  test  priorities  overtaxed  available  operational 
test  resources  needed  to  support  planned  FY15  FOT&E.  This 
FOT&E  is  planned  to  begin  in  2QFY16. 

•  The  Air  Force  originally  intended  to  fulfill  the  MQ-9 
Increment  1  Capability  Production  Document  requirements 
with  a  final  UAS  configuration  consisting  of  the  Block 

5  RPA,  Block  30  GCS,  and  OFP  904.6.  AFOTEC  will 
conduct  FOT&E  of  this  configuration  when  ongoing  thermal 
management  problems  are  resolved.  Subsequent  AFOTEC 
FOT&E  will  be  required  consistent  with  the  evolving  MQ-9 
program  of  record  content,  and  at  a  minimum  will  include 
evaluation  of  the  planned  Block  50  GCS  in  conjunction  with 
the  appropriate  system  capabilities  being  delivered  under  the 
Air  Force  FY15  hybrid  acquisition  strategy. 

•  Electromagnetic  Environmental  Effects  testing  on  the  Block 
5  RPA  will  not  take  place  until  after  completion  of  planned 
FY16  FOT&E.  Such  testing  recently  completed  on  the 
Block  1  aircraft  revealed  vulnerabilities,  and  due  to  the 
similarities  between  the  Block  1  and  Block  5  aircraft,  similar 
vulnerabilities  may  exist  on  the  Block  5  aircraft. 

•  The  current  DOT&E-approved  MQ-9  TEMP  supports  FOT&E 
of  the  Block  5  RPA,  Block  30  GCS,  and  OFP  904.6.  Upon 
completion  of  FOT&E  for  this  configuration,  a  new  TEMP 
will  be  required  to  document  the  incorporation  of  new 
program  of  record  content  (e.g.,  the  Block  50  GCS)  into  the 
MQ-9  Increment  1  system. 
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Recommendations 

•  Status  of  Previous  Recommendations.  In  FY 15,  the  Air  Force 
completed  the  development  of  the  MQ-9  UAS  to  support 
planned  FOT&E  of  the  Block  5  RPA,  Block  30  GCS,  and 
OFP  904.6.  The  Air  Force  made  progress  toward,  but  did  not 
fully  satisfy,  the  FY14  recommendation  to  resolve  the  hot 
weather  operating  shortfalls. 

•  FY  15  Recommendations.  The  Air  Force  should: 

1 .  Resolve  the  remaining  Block  5  RPA  thermal  and  power 
management  shortfalls  prior  to  the  start  of  AFOTEC’s 
FOT&E. 


2.  Complete  planned  FOT&E  of  the  Block  5  RPA, 

Block  30  GCS,  and  OFP  904.6. 

3.  Upon  completion  of  the  planned  FOT&E  (Block  5  RPA, 
Block  30  GCS,  and  OFP  904.6),  submit  a  new  TEMP 
documenting  the  incorporation  of  new  content  and 
capabilities  (e.g.,  Block  50  GCS)  and  the  T&E  strategy  and 
resources  required  to  mature  and  test  these  capabilities. 
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QF-16  Full-Scale  Aerial  Target  (FSAT) 


Executive  Summary 

•  The  Air  Force  conducted  99  QF-16  developmental  and 
operational  test  sorties  in  FY13  and  14,  completing  IOT&E  in 
September  2014.  DOT&E’s  QF-16  IOT&E  report  submitted 
in  January  2015  stated  that  the  QF-16  is  operationally 
effective  and  will  provide  a  high-fidelity  representation  of 
fourth-generation  air  superiority  threats  for  U.S.  weapon 
systems  testing  and  tactics,  techniques,  and  procedures 
development.  Additionally,  the  QF-16  is  operationally 
suitable,  and  under  the  53rd  Weapons  Evaluation  Group 
Concept  of  Employment  that  provides  a  primary  and  spare 
target  for  each  mission,  the  QF-16  achieved  98.3  percent 
mission  supportability.  Although  there  are  deficiencies  that 
prevent  the  QF-16  from  meeting  its  Mission  Supportability 
and  Mean  Time  Between  Failure  (MTBF)  requirements,  they 
do  not  have  significant  operational  impact.  The  program  is 
not  required  to  meet  the  MTBF  requirement  until  the  QF-16 
reaches  Full  Operational  Capability. 

•  The  Air  Force  should  accomplish  cybersecurity  testing 
in  accordance  with  the  DOT&E  cybersecurity  policy 
memorandum,  dated  August  1,  2014. 

•  The  Air  Force  should  ensure  procurement  funding  provides 
at  least  25  Full-Scale  Aerial  Target  (FSAT)  targets  per  year 
beginning  in  FY16  to  meet  Service-coordinated  aerial  target 
requirements,  in  compliance  with  Resource  Management 
Decision  700. 

•  The  Air  Force  should  provide  plans  for  Phase  II  of  the  Air 
Superiority  Target  program  to  address  shortfalls  in  testing 
against  fifth-generation  airborne  threats. 

System 

•  The  QF-16  is  the  latest  FSAT  designed  to  test  and  evaluate 
U.S.  weapon  systems  and  assist  in  developing  tactics, 
techniques,  and  procedures  to  counter  fighter-size  airborne 
threats.  The  DOD  is  replacing  the  current  FSAT,  the  QF-4, 
due  to  its  increasing  dissimilarity  from  current  and  projected 
air-superiority  threats,  declining  supportability,  and  depletion 
of  suitable  F-4  airframes. 

•  The  QF-16  system  is  composed  of  regenerated  F-16  Block  15, 
25,  and  30  aircraft  equipped  with  Drone-Peculiar  Equipment 
to  enable  remote  command  and  control,  missile  trajectory 


scoring,  and  safe  flight  termination.  Like  the  QF-4,  the  QF-16 
is  capable  of  manned  and  Not  Under  Live  Local  Operator 
flight  operations.  It  will  operate  from  Tyndall  AFB,  Florida, 
using  the  Gulf  Range  Drone  Control  System,  and  Holloman 
AFB,  New  Mexico,  using  the  White  Sands  Integrated  Target 
System  located  at  White  Sands  Missile  Range,  New  Mexico. 

•  The  QF-16  retains  F-16  flight  performance  characteristics 
and  payload  capabilities  including  supersonic,  after-burning 
engines,  high-G  maneuvering,  complex  electronic  attack,  and 
expendable  countermeasures. 

Mission 

The  DOD  uses  FSATs  to: 

•  Provide  threat-representative  presentations  for  developmental 
and  operational  test  and  evaluation  for  U.S.  weapon  systems, 
as  mandated  by  Title  10  U.S.  Code,  Section  2366 

•  Continuously  evaluate  fielded  air-to-air  missile  capabilities 
while  providing  live  missile  training  for  combat  air  crews 
through  Air  Force  and  Navy  Weapon  Systems  Evaluation 
Programs 

Major  Contractor 

The  Boeing  Company  -  St.  Louis,  Missouri 


Activity 

•  In  January  2015,  DOT&E  submitted  an  IOT&E  report  on  the 
QF-16  FSAT. 

•  On  September  5,  2014,  QF-16  completed  its  last  IOT&E 
flight.  In  total,  QF-16  flew  99  developmental  and  operational 
test  sorties,  19  of  which  were  integrated  test,  and  4  dedicated 


operational  test  sorties  (2  manned  and  2  Not  Under  Live  Local 
Operator).  The  Air  Force  conducted  the  IOT&E  in  accordance 
with  the  DOT&E-approved  test  plan  dated  May  31,  2013. 


QF-16  FSAT  349 


Assessment 

•  As  detailed  in  the  IOT&E  report,  DOT&E  found  that 
the  QF-16  is  operationally  effective  and  will  provide  a 
high-fidelity  representation  of  fourth-generation  air  superiority 
threats  for  U.S.  weapon  systems  testing  and  tactics,  techniques, 
and  procedures  development. 

-  The  QF-16  demonstrated  the  full  Mission  Performance 
Key  Performance  Parameter  (KPP)  profile  with  the  Target 
Control  Systems  at  Tyndall  AFB,  Florida,  and  White  Sands 
Mission  Range,  New  Mexico.  The  Mission  Performance 
KPP  profile  included  ground  operations,  launch  and  climb 
out,  flight  maneuvers,  post-shot  procedures,  and  recovery. 

-  The  QF-16  successfully  demonstrated  the  ability  to 
carry,  operate,  and  monitor  all  KPP-required  mission 
pods  used  for  countermeasures  and  electronic  attack, 
including  the  Tactical  Electronic  Attack  Missions  System, 
ALQ-167,  ALQ-188,  ALE-56,  and  ALE-40.  The  QF-16 
flew  with  AIM-9  sized  pods  and  a  centerline  fuel  tank, 
and  successfully  employed  both  internal  and  external 
expendables. 

-  The  QF-16  successfully  demonstrated  the  Flight 
Termination  System  and  Vector  Scoring  System  (VSS), 
which  provides  miss  distance  between  a  missile  and  the 
QF-16. 

•  As  reported  in  the  IOT&E  report,  DOT&E  found  that 
the  QF-16  is  operationally  suitable.  Although  there  are 
deficiencies  that  prevent  the  QF-16  from  meeting  its  Mission 
Supportability  and  MTBF  requirements,  they  do  not  have 
significant  operational  impact. 

-  Under  the  53rd  Weapons  Evaluation  Group  Concept  of 
Employment  that  provides  a  primary  and  a  spare  target  for 
each  mission,  the  QF-16  achieved  98.3  percent  mission 
supportability. 

-  Calculations  from  test  data  measure  MTBF  to  be 
31.1  hours  with  80  percent  confidence  bounds  of 
23.74  hours  and  41.21,  short  of  the  45 -hour  Full 
Operational  Capacity  requirement.  About  half  (12  of  25) 
of  QF-16  drone-peculiar  equipment  failures  were  related  to 
the  VSS. 

-  The  drone-unique  Mean  Time  to  Repair  is  28.4  ±17.2 
minutes,  well  within  the  requirement  of  less  than 

90  minutes  for  events  within  the  90th  percentile.  Training 
and  technical  orders  are  adequate  for  both  maintainers  and 
operators. 

•  Initially,  DOT&E  exempted  QF-16  from  a  cybersecurity 
Cooperative  Vulnerability  Penetration  Assessment  and 
Adversarial  Assessment  because  the  system  had  no  direct 
connection  to  the  internet  and  had  a  Flight  Termination 
System  independent  of  the  control  system.  However, 

DOT&E  published  refined  cybersecurity  testing  guidance 
for  acquisition  programs  in  August  2014.  The  Air  Force 
Life  Cycle  Management  Center,  with  the  support  of  the 
Air  Force  Operational  Test  and  Evaluation  Center,  is  in  the 
process  of  test  planning  to  comply  with  DOT&E ’s  additional 
cybersecurity  testing  requirements.  The  QF-16  program 
currently  lacks  funding  for  additional  cybersecurity  testing. 


•  The  Air  Force  did  not  require  QF-16  to  represent 
fifth-generation  airborne  threat  systems  (including  radio 
frequency  low-observability  characteristics,  internally-carried 
advanced  electronic  attack,  and  low  probability  of  intercept 
sensors).  DOT&E  continues  to  emphasize  existing  aerial 
targets,  including  the  QF-16,  are  insufficient  for  adequate 
operational  testing  of  U.S.  weapon  systems. 

-  In  the  Air  Superiority  Target  Phase  1  Analysis  of 
Alternatives  Final  Report  (March  15,  2007),  the  Air  Force 
recommended  further  study  to  produce  user  consensus 

on  critical  characteristics  of  future  aerial  targets  and 
to  determine  capabilities  and  shortfalls  in  existing  test 
resources. 

-  Multiple  stakeholders  within  Congress,  OSD,  the  Air 
Force,  and  the  Navy,  support  the  requirement  for  a 
fifth  generation  FSAT. 

•  In  late  June,  Boeing  performed  sample  inspections  on  a 
QF-16  at  Cecil  Field,  Florida,  and  discovered  workmanship 
deficiencies  with  wire  splices,  termination,  and  routing.  As 
a  result  of  these  findings,  they  broadened  the  inspection 
population  to  the  first  three  production  aircraft  already 
delivered  to  Tyndall  AFB,  Florida,  and  found  similar  problems. 
Corrective  actions  are  ongoing  and  Air  Combat  Command  is 
expected  to  declare  Initial  Operational  Capability  in  FY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  still 
needs  to  address  the  following  outstanding  recommendations: 

1.  Complete  radar  cross  section  measurements  for  QF-16  to 
ensure  current  and  future  U.S.  weapon  systems  programs 
have  precise,  reliable  data  on  system  performance  against 
measured,  low-observable  target  presentations.  Of  note, 
the  program  is  on  pace  to  complete  all  of  the  necessary 
measurements  by  February  2017. 

2.  Ensure  QF-16  procurement  funding  continues  to  comply 
with  Resource  Memorandum  Decision  700-mandated  levels 
of  25  aircraft  per  year  beginning  in  FY16,  in  order  to  meet 
Service-coordinated  and  approved  test  and  training  resource 
requirements. 

3.  Complete  the  user  requirements  and  current  capabilities 
studies  and  provide  plans  for  Phase  II  of  the  Air  Superiority 
Target  program  to  address  test  and  evaluation  shortfalls  for 
U.S.  weapon  systems  with  respect  to  threat-representative, 
fifth-generation  FSATs. 

•  FY 15  Recommendations.  The  Air  Force  should: 

1 .  Accomplish  cybersecurity  testing  in  accordance  with 
the  DOT&E  cybersecurity  policy  memorandum,  dated 
August  1,  2014. 

2.  Continue  to  monitor  VSS  reliability  to  ensure  the 
corrections  that  were  implemented  in  low-rate  initial 
production  Lot  1  aircraft  will  support  compliance  with  the 
QF-16’s  MTBF  requirement. 
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RQ-4B  Global  Hawk  High-Altitude  Long-Endurance 
Unmanned  Aerial  System  (UAS) 


Executive  Summary 

•  In  F ebruary  20 1 5,  U SD( AT &L)  approved  a  multi-year, 

$3.5  Billion  upgrade  and  modernization  development  program 
to  accomplish  the  following:  correct  previous  capability 
shortfalls  identified  during  the  2011  RQ-4B  Block  30  IOT&E; 
address  emerging  component  obsolescence  problems; 
and  significantly  upgrade  system  sensor,  ground  station, 
and  communication  systems.  The  Air  Force  is  currently 
developing  a  comprehensive  program  test  strategy  and  Test 
and  Evaluation  Master  Plan  (TEMP)  to  guide  development 
and  testing  for  the  extensive  system  architecture  and  capability 
upgrades  included  in  the  new  program  baseline  and  future 
modernization  programs. 

•  The  Air  Force  is  currently  planning  to  conduct  RQ-4B 
Block  30/Airborne  Signals  Intelligence  Payload  (ASIP) 
FOT&E  in  conjunction  with  the  initial  phases  of  the  RQ-4B 
modernization  program  in  FY18.  This  test  should  include 

a  complete  re-evaluation  of  the  RQ-4B  Block  30  Signals 
Intelligence  (SIGINT)  mission  capabilities  with  the  ASIP 
sensor  as  well  as  an  assessment  of  previously  identified  ground 
station,  air  vehicle,  communication  system,  interoperability, 
and  cybersecurity  shortfalls. 

•  Following  numerous  developmental  test  delays,  the  RQ-4B 
Block  40/Multi  Platform  Radar  Technology  Insertion  Program 
(MP-RTIP)  IOT&E  began  in  September  2015  and  completed 
in  December  2015.  The  Air  Force  conducted  testing  in 
accordance  with  the  DOT&E-approved  test  plan. 

System 

•  The  RQ-4B  Global  Hawk  is  a  remotely-piloted,  high 
altitude,  long-endurance  airborne  Intelligence,  Surveillance, 
and  Reconnaissance  (ISR)  system  that  includes  the  Global 
Hawk  unmanned  air  vehicle,  various  intelligence  and 
communications  relay  mission  payloads,  and  supporting 
command  and  control  ground  stations. 

•  The  RQ-4B  Global  Hawk  Block  30  system  is  equipped  with 
a  multi-intelligence  payload  that  includes  both  the  Enhanced 
Integrated  Sensor  Suite  imagery  intelligence  payload  and  ASIP 
SIGINT  sensor. 

•  The  RQ-4B  Block  40  system  is  equipped  with  the  MP-RTIP 
synthetic  aperture  radar  payload  designed  to  simultaneously 
collect  imagery  intelligence  on  stationary  ground  targets  and 
track  ground-moving  targets. 

•  All  RQ-4B  systems  use  line-of-sight  and  beyond-line-of-sight 
communication  systems  to  provide  air  vehicle  command  and 
control  and  transfer  collected  intelligence  data  to  grounds 
stations  for  exploitation  and  dissemination. 

•  The  Air  Force  Distributed  Common  Ground  System 

(AF  DCGS)  supports  ISR  collection,  processing,  exploitation. 


analysis,  and  dissemination  for  both  Block  30  and  40  RQ-4B 
Global  Hawk  systems.  The  AF  DCGS  employs  global 
communications  architecture  to  connect  multiple  intelligence 
platforms  and  sensors  to  numerous  Distributed  Ground 
Stations  where  intelligence  analysts  produce  and  disseminate 
intelligence  products. 

Mission 

•  Commanders  use  RQ-4  Global  Hawk  reconnaissance  units  to 
provide  high-altitude,  long-endurance  intelligence  collection 
capabilities  to  support  theater  operations. 

•  Operators  collect  imagery  and  SIGNINT  data  to  support 
ground  units  and  to  identify  intelligence-essential  elements 
of  information  for  theater  commanders.  Units  equipped 
with  RQ-4B  Global  Hawk  use  line-of-sight  and  beyond 
line-of-sight  satellite  datalinks  to  control  the  Global  Hawk 
system  and  transmit  collected  intelligence  data. 

•  Ground-based  intelligence  analysts  exploit  collected  imagery, 
ground-moving  target,  and  SIGINT  to  provide  intelligence 
products  that  support  theater  operations. 

•  Global  Hawk  can  also  provide  imagery  intelligence  directly  to 
forward-based  personnel  through  direct  line-of-sight  datalink 
systems. 

Major  Contractor 

Northrop  Grumman  Aerospace  Systems,  Strike  and  Surveillance 

Systems  Division  -  San  Diego,  California 
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Activity 

•  The  2015  Presidential  Budget  fully  funded  the  Global  Hawk 
program,  resolving  several  years  of  programmatic  uncertainty. 
As  of  September  2015,  the  Air  Force  has  taken  delivery  of 

18  of  21  RQ-4B  Block  30  air  vehicles  and  all  11  RQ-4B 
Block  40  air  vehicles,  along  with  9  Mission  Control  and 
10  Launch  and  Recovery  ground  stations. 

•  In  F ebruary  20 1 5,  U SD( AT &L)  approved  a  multi-year, 

$3.5  Billion  upgrade  and  modernization  development  program 
to  accomplish  the  following:  correct  previous  capability 
shortfalls  identified  during  the  2011  RQ-4B  Block  30  IOT&E; 
address  emerging  component  obsolescence  problems;  and 
significantly  upgrade  system  sensor,  ground  station,  and 
communication  systems. 

•  The  Air  Force  is  currently  developing  a  comprehensive 
program  test  strategy  and  TEMP  to  guide  development  and 
testing  for  the  extensive  system  architecture  and  capability 
upgrades  included  in  the  new  program  baseline  and  future 
modernization  programs. 

Block  30 

•  No  RQ-4B  Block  30  operational  testing  was  conducted  in 
FY15.  The  Air  Force  continued  to  sustain  operations  for 

18  Block  30  aircraft  at  Beale  AFB,  California,  and  at  forward 
operating  bases  in  U.S.  Pacific  Command  (USPACOM), 

U.S.  Central  Command  (USCENTCOM),  and  U.S.  European 
Command  (USEUCOM)  operating  areas. 

•  The  Air  Force  is  currently  developing  a  comprehensive 
program  test  strategy  and  TEMP  to  correct  previously 
identified  RQ-4B  Block  30  capability  shortfalls  and  test  a 
series  of  other  modernization  upgrades.  This  strategy  will 
identify  the  next  collection  of  significant  RQ-4B  Block 
30  FOT&E  events  planned  for  FY18.  Events  include 
re-evaluation  of  previously  identified  ASIP/SIGINT  mission 
capability  shortfalls,  interoperability  deficiencies,  MS- 177 
sensor  integration,  weather  radar  performance,  mission 
planning  upgrades,  and  other  system  modernization  changes. 
Block  40 

•  In  FY15,  the  Air  Force  continued  to  employ  two  RQ-4B  Block 
40  development  systems  with  limited  operational  capabilities 
in  the  USCENTCOM  area  of  operations.  Two  additional 
systems  are  deployed  in  the  USPACOM  area  of  operations. 
These  systems  were  fielded  in  FY13  and  FY14  to  support 
Combatant  Command  requests  for  additional  airborne  ISR 
support. 

•  Following  numerous  developmental  test  delays,  the  RQ-4B 
Block  40/MP-RTIP  IOT&E  began  in  September  2015,  and 
completed  in  December  2015.  The  Air  Force  conducted 
testing  in  accordance  with  the  DOT&E-approved  test  plan. 
Initial  schedule  delays  were  associated  with  synthetic  aperture 
radar  image  quality  problems  and  system  stability  problems 
observed  during  the  FY13  RQ-4B  Block  40  Operational 
Utility  Evaluation  (OUE).  Delayed  delivery  of  AF  DCGS 
system  software  changes  necessary  to  support  RQ-4B  Block 
40  operations  further  delayed  the  start  of  IOT&E. 

•  DOT&E  approved  the  Block  40  IOT&E  test  plan  in 
May  2015. 
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Assessment 

Block  30 

•  Since  the  combined  RQ-4B  Block  30/ASIP  IOT&E  in  2011, 
the  Air  Force  has  corrected  most  RQ-4B  air  vehicle  reliability 
and  availability  problems  and  implemented  a  limited  number 
of  previously  planned  system  improvements.  However,  due 
to  the  program  uncertainty  driven  by  the  FY13  DOD  decision 
to  retire  the  RQ-4B  fleet,  and  the  subsequent  reversal  of  that 
decision,  the  Air  Force  has  not  yet  conducted  a  comprehensive 
FOT&E  to  verify  correction  of  all  major  IOT&E  deficiencies. 
Currently  fielded  RQ-4B  Block  30  systems  continue  to 
operate  with  many  of  the  same  operational  performance, 
interoperability,  and  SIGINT  mission  deficiencies  identified 
during  IOT&E. 

•  The  Air  Force  is  currently  planning  to  conduct  FOT&E 
in  conjunction  with  the  initial  phases  of  the  RQ-4B 
modernization  program  in  FY18.  This  test  should  include 
a  complete  re-evaluation  of  the  RQ-4B  Block  30  SIGINT 
mission  capabilities  with  the  ASIP  sensor  as  well  as  an 
assessment  of  previously  identified  ground  station,  air  vehicle, 
communication  system,  interoperability,  and  cybersecurity 
shortfalls. 

Block  40 

•  Since  the  FY13  RQ-4B  Block  40  OUE,  the  Air  Force 
has  implemented  a  series  of  software  changes  to  improve 
MP-RTIP  sensor  stability  and  performance.  Field  data  from 
USCENTCOM  early  fielding  activities  indicate  that  software 
changes  and  procedural  workarounds  have  improved  sensor 
stability. 

•  IOT&E  began  in  September  2015  and  included  10  sorties  that 
concluded  in  October  2015;  additional  data  collection  and 
analysis  continued  until  the  end  of  December  2015. 

•  DOT&E  intends  to  submit  a  report  on  the  RQ-4B  Block  40 
IOT&E  in  2QFY16. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force  made 
limited  progress  toward  addressing  FY 14  recommendations. 
The  Air  Force  has  not  completed  an  RQ-4B  Block  30 

or  RQ-4B  Block  40  TEMP  to  guide  developmental  and 
operational  testing  of  these  systems  or  proceeded  with  an  ASIP 
sensor  FOT&E  event  to  verify  correction  of  performance 
deficiencies  identified  during  the  2011  RQ-4B  Block  30 
IOT&E.  The  Air  Force  has  identified  and  corrected  some 
persistent  RQ-4B  Block  40/MP-RTIP  sensor  stability 
problems,  but  has  not  verified  whether  these  corrections 
will  provide  enough  improvement  to  correct  the  significant 
operational  performance  shortfalls  that  have  been  identified 
since  the  initial  AFOTEC  MP-RTIP  operational  assessment  in 
2008.  The  Air  Force  did  implement  corrective  actions  for  the 
MP-RTIP  synthetic  aperture  radar  image  resolution  problems 
observed  during  the  2013  RQ-4B  Block  40  OUE. 


FY15  Recommendations.  The  Air  Force  should:  2.  Develop  a  plan  to  complete  the  FOT&E  for  the  RQ-4B 

1.  Develop  an  RQ-4B  program  TEMP  to  guide  completion  of  Block  30  SIG1NT  mission  using  the  ASIP  sensor. 

post-IOT&E  corrective  actions  and  to  define  operational 
test  requirements  for  future  Block  30  and  Block  40  system 
upgrades. 
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Small  Diameter  Bomb  (SDB)  II 


Executive  Summary 

•  The  Air  Force  Operational  Test  and  Evaluation  Center  and  the 
Navy’s  Commander,  Operational  Test  and  Evaluation  Force, 
completed  the  Small  Diameter  Bomb  (SDB)  II  Operational 
Assessment  in  February  2015. 

•  DOT&E  published  an  Operational  Assessment  report  in 

May  2015  to  support  a  Milestone  C  (MS  C)  decision  and  entry 
into  low-rate  initial  production. 

•  The  Milestone  Decision  Authority  approved  the  SDB  II  MS  C 
in  May  2015.  Subsequently,  the  Program  Office  awarded  the 
Lot  1  production  option. 

•  The  SDB  II  program  test  team  is  completing  developmental 
flight  testing.  The  program  has  accomplished  12  of  the 

16  Normal  Attack  (NA)  mode  Guided  Test  Vehicle  (GTV) 
shots  and  3  of  the  6  NA  Live  Fire  (LF)  shots  required  to  enter 
Government  Confidence  Testing  (GCT).  SDB  II  is  on  track  to 
begin  GCT  in  early  2016. 

•  The  weapon  is  progressing  well  towards  demonstrating  its 
requirements  in  the  NA  mode.  It  has  demonstrated  in-flight 
target  updates  with  both  Ultra  High  Frequency  and  Link  16 
networks. 

System 

•  The  SDB  II  is  a  250-pound,  air-launched,  precision-glide 
weapon  that  uses  deployable  wings  to  achieve  stand-off 
range.  F-15E  aircraft  employ  SDB  IIs  from  the  BRU-61/A 
four-weapon  carriage  assembly. 

•  SDB  II  combines  Millimeter- Wave  radar,  imaging  infrared, 
and  laser-guidance  sensors  in  a  terminal  seeker,  in  addition  to 
a  GPS  and  an  Inertial  Navigation  System  to  achieve  precise 
guidance  accuracy  in  adverse  weather. 

•  The  SDB  II  incorporates  a  multi-function  (blast, 
fragmentation,  and  shaped  charged  jet)  warhead,  designed  to 
defeat  non-armored  and  armored  targets.  The  weapon  can  be 
set  to  initiate  on  impact,  at  a  preset  height  above  the  intended 
target,  or  in  a  delayed  mode. 

•  SDB  II  provides  increased  weapons  load  per  aircraft  compared 
to  legacy  air-to-ground  munitions  used  against  offensive 


counter-air,  strategic  attack,  interdiction,  and  close  air  support 
targets  in  adverse  weather. 

•  SDB  II  provides  reduced  collateral  damage  while  achieving 
kills  across  a  broad  range  of  target  sets  by  precise  accuracy, 
small  warhead  design,  and  focused  warhead  effects. 

•  There  are  three  principal  attack  modes:  NA,  Laser  Illuminated 
Attack  (LIA),  and  Coordinate  Attack  (CA).  SDB  II  can 

be  used  against  moving  or  stationary  targets  using  its  NA 
(radar/infrared  sensors)  or  LIA  modes,  and  fixed  targets  with 
its  CA  mode. 

Mission 

•  Combatant  Commanders  will  use  units  equipped  with  SDB  II 
to  attack  stationary  and  moving  targets  in  degraded  weather 
conditions  at  stand-off  ranges. 

•  An  SDB  II-equipped  unit  or  Joint  Terminal  Attack  Controller 
will  engage  targets  in  dynamic  situations  and  use  a  weapon 
datalink  network  to  provide  in-flight  target  updates,  in-flight 
retargeting,  weapon  in-flight  tracking,  and,  if  required,  weapon 
abort. 

Major  Contractor 

Raytheon  Missile  Systems  -  Tucson,  Arizona 


Activity 

•  The  Air  Force  Operational  Test  and  Evaluation  Center  and  the 
Navy’s  Commander,  Operational  Test  and  Evaluation  Force, 
completed  the  SDB  II  Operational  Assessment  in  February 
2015. 

•  DOT&E  published  an  Operational  Assessment  report  in 

May  2015  to  support  a  Milestone  C  (MS  C)  decision  and  entry 
into  low-rate  initial  production. 


•  The  Milestone  Decision  Authority  approved  the  SDB  II  MS 
C  in  May  2015;  the  SDB  II  Program  Office  awarded  the  Lot  1 
production  option  for  144  Air  Force  weapons  in  June  2015. 

•  DOT&E  approved  the  updated  Test  and  Evaluation  Master 
Plan  in  April  2015;  the  SDB  II  Program  Office  is  preparing  for 
IOT&E. 
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•  As  of  2015,  the  Air  Force  has  successfully  completed  12  GTV 
and  3  LF  developmental  tests  against  moving  and  stationary 
targets.  Three  of  the  GTV  and  one  LF  test  were  conducted 
with  Ultra  High  Frequency  updates  while  nine  GTV  and 

two  LF  test  shots  were  conducted  with  Link  16  updates.  All 
tests  were  conducted  in  the  NA  mode,  which  is  the  primary 
employment  method  for  SDB  II.  LIA  and  CA  will  be  tested  in 
2QFY16. 

•  The  Program  Office  completed  12  rounds  of  seeker  CFTs, 
resulting  in  over  1,833  target  runs  in  a  wide  variety  of  terrain 
and  environmental  conditions  providing  terabytes  of  seeker 
performance  data  and  over  439  hours  of  seeker  operation 
without  a  single  failure. 

•  The  program  has  augmented  and  refined  the  Integrated  Flight 
System  (IFS)  model  by  incorporating  the  results  of  over 
1,833  CFT  runs  as  well  as  weapon  flight  tests.  IFS  model 
verification  and  validation  is  expected  to  be  complete  by  the 
end  of  calendar  year  2015,  and  the  Air  Force  Operational  Test 
and  Evaluation  Center  is  expected  to  accredit  it  prior  to  the 
start  of  operational  testing. 

•  The  Program  Office  completed  over  2,000  hours  of  ground 
reliability  testing  and  nearly  200  hours  of  inflight  reliability 
testing. 

•  The  program  is  scheduled  to  begin  a  28-shot  NA  mode  GCT 
program  in  January  2016,  which  will  test  the  weapon  in 
more  operationally  realistic  environments  with  operationally 
representative  hardware  and  software.  GCT  will  test  the 
weapon  versus  maritime  targets,  countermeasures,  and 
GPS-degraded  environments. 

Assessment 

•  The  operational  assessment  showed  the  SDB  II  progressing 
well  towards  meeting  its  effectiveness,  reliability,  and 
lethality  requirements  in  the  NAmode,  which  is  the  primary 
employment  method  for  SDB  II.  SDB  II  successfully  engaged 
both  moving  and  stationary  targets,  in  which  there  were  no 
free  flight  reliability  failures.  The  program  has  implemented 
corrective  actions  and  fixes  for  all  failure  modes  discovered  in 
test. 


•  The  SDB  II  program  met  the  MS  C  criteria.  The  SDB  II 
Program  Office  is  preparing  for  IOT&E  with  an  adequately 
resourced  test  program  and  no  unresolved  major  programmatic 
testing  problems.  IOT&E  is  scheduled  to  begin  in  2QFY17. 

•  Two  GTVs  and  one  LF  mission  required  additional  attempts 
and  were  successfully  repeated  after  completion  of  the  failure 
investigation  and  implementation  of  corrective  actions.  All 
corrective  actions  to  date  have  been  successful  in  preventing 
repeats  of  the  observed  failure  modes,  with  the  exception  of 
LF-5.  LF-5,  which  was  conducted  on  September  14,  2015, 
did  not  detonate  and  is  currently  under  investigation.  The  test 
will  be  repeated  after  the  investigation  to  adequately  assess  the 
lethality  of  the  SDB  II  against  the  specific  target. 

•  The  weapon  failed  one  environmental  test  related  to  the 
shipboard  environment,  but  the  program  implemented  a  fix 
and  completed  design  verification  testing  giving  the  Program 
Office  confidence  that  the  corrective  actions  will  be  successful. 

•  Preliminary  comparisons  of  the  IFS  model  pre-  and  post-flight 
predictions  indicate  the  model  is  adequate  for  the  kinematics 
flown  in  flight  test  to  date.  Raytheon  Missile  Systems  will 
continue  to  develop  and  update  the  IFS  model,  which  will 

be  essential  to  the  assessment  of  the  results  of  live  fire  and 
operational  testing.  IFS,  in  combination  with  lethality  data, 
will  produce  single  shot  kill  probability  values  needed  to 
assess  end-to-end  weapon  effectiveness  against  a  range  of 
operationally  relevant  targets. 

•  The  weapon  is  progressing  towards  demonstrating  the  required 
reliability.  Further  testing  in  GCT  and  CFT  should  provide 
increased  confidence  in  weapon  reliability. 

•  The  weapon  is  on  track  to  proceed  to  GCT. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  Air  Force 
completed  all  previous  recommendations. 

•  FY15  Recommendation. 

1.  The  Air  Force  should  use  the  results  of  GCT  to  further 
refine  the  IOT&E  test  plan. 
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Ballistic  Missile  Defense  System  (BMDS) 
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[ _ I  Future  Enhancement  □  Non-Missile  Defense  Agency  Element 


Executive  Summary 

•  The  Flight  Test,  Operational-02  (FTO-02)  Event  2  and 
Event  2a  test  demonstrated  that  the  Ballistic  Missile  Defense 
System  (BMDS)  capability  against  theater/regional  threats 
increased  during  FY 15.  The  BMDS  demonstrated  layered 
defense  against  two  threat-representative  ballistic  missile 
targets.  However,  the  Standard  Missile  (SM)-3  Block 

IB  Threat  Update  guided  missile  failed  early  in  flight;  an 
engineering  Failure  Review  Board  investigation  is  underway. 
Full  assessment  of  the  FTO-02  Event  2  and  Event  2a  test 
mission  data  with  respect  to  BMDS  operational  effectiveness, 
operational  suitability,  and  interoperability  is  ongoing. 

•  With  no  flight  testing  in  FY  15,  the  Homeland  Defense 
assessment  remains  unchanged.  The  Ground-based  Midcourse 
Defense  (GMD)  combat  system  demonstrates  a  limited 
capability  to  defend  the  U.S.  Homeland  from  small  numbers 


of  intermediate-range  or  intercontinental  ballistic  missile 
threats  launched  from  North  Korea  or  Iran. 

•  The  Missile  Defense  Agency  (MDA)  enhanced  the  Integrated 
Master  Test  Plan  (IMTP)  by  including  direct  linkage  between 
the  BMDS  test  program  and  future  capability  enhancements. 

•  The  MDA  conducted  numerous  ground  tests,  wargames, 
and  exercises.  The  capability  to  produce  BMDS-level 
simulation-based  performance  assessments  was  limited.  The 
MDA  should  increase  the  development  priority  and  associated 
funding  for  the  BMDS  high-fidelity,  end-to-end,  digital 
modeling  and  statistically  significant  simulation  capability. 

System 

The  BMDS  is  a  distributed  system  currently  including  five 

elements:  four  autonomous  combat  systems  including  dedicated 
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sensors  and  one  global/regional  sensor/command  and  control 

architecture. 

•  GMD  (shooter),  COBRA  DANE  Radar  Upgrade  (sensor), 
Upgraded  Early  Warning  Radars  (sensor),  and  Sea-Based 
X-band  Radar  (sensor). 

•  Aegis  BMD  (shooter),  Aegis  Ashore  Missile  Defense  System 
(AAMDS)  (shooter),  and  AN/SPY- 1  Radar  (sensor). 

•  Terminal  High- Altitude  Area  Defense  (THAAD)  (shooter)  and 
AN/TPY-2  Terminal  Mode  Radar  (sensor). 

•  Patriot  (shooter).  Since  Patriot  is  an  Army  program  instead 
of  an  MDA  program,  testing  of  the  Patriot  is  discussed  in  the 
Army  section  of  this  FY15  Annual  Report. 

•  Space-Based  Infrared  System/Defense  Support  Program 
(SBIRS/DSP)  (global  sensor);  AN/TPY-2  Forward-Based 
Mode  (FBM)  Radar  (regional  sensor);  and  Command  and 
Control,  Battle  Management,  and  Communications  (C2BMC) 
(command  and  control).  SB1RS  and  DSP  are  currently  Air 
Force  operational  assets.  Due  to  hardware  and  software 
similarity  to  the  AN/TPY-2  (Terminal  Mode)  Radar,  testing  of 
the  AN/TPY-2  (FBM)  Radar  is  covered  in  the  THAAD  annual 
report. 

Mission 

•  U.S.  Northern  Command  (USNORTHCOM),  U.S.  Pacific 
Command  (USPACOM),  U.S.  European  Command 
(USEUCOM),  and  U.S.  Central  Command  (USCENTCOM) 
employ  the  assets  of  the  BMDS  to  defend  U.S.  territory, 
deployed  forces,  and  allies  against  ballistic  missile  threats  of 
all  ranges. 

•  The  U.S.  Strategic  Command  (USSTRATCOM)  synchronizes 
operational-level  global  missile  defense  planning  and 
operations  support  for  the  DOD. 


•  All  Combatant  Commanders  use  the  C2BMC  element  of 
the  BMDS  to  maintain  situational  awareness.  USEUCOM, 
USCENTCOM,  and  USPACOM  also  use  the  C2BMC  to 
provide  sensor  management  of  regional  AN/TPY-2  (FBM) 
radars. 

Major  Contractors 

•  The  Boeing  Company 

-  GMD  Integration  -  Huntsville,  Alabama 

•  Lockheed  Martin  Corporation 

-  Aegis  BMD,  AAMDS,  and  AN/SPY- 1 
Radar  -  Moorestown,  New  Jersey 

-  C2BMC  -  Gaithersburg,  Maryland 

-  SB1RS  -  Sunnyvale,  California 

-  THAAD  Weapon  System  and  Patriot  Interceptors  -  Dallas, 
Texas 

-  THAAD  Interceptors  -  Troy,  Alabama 

•  Northrop  Grumman  Corporation 

-  DSP  -  Redondo  Beach,  California 

-  GMD  Fire  Control  and  Communications  -  Huntsville, 
Alabama 

•  Orbital  Sciences  Corporation 

-  GMD  Booster  Vehicles  -  Chandler,  Arizona 

•  Raytheon  Company 

-  GMD  Exo-atmospheric  Kill  Vehicle  and  Standard 
Missile-3  Interceptors  -  Tucson,  Arizona 

-  Patriot,  AN/TPY-2  Radar,  Cobra  Dane  Radar, 

Sea-Based  X-band  Radar,  and  Upgraded  Early  Warning 
Radars  -  Tewksbury,  Massachusetts 


Activity 

•  The  MDA  conducted  all  testing  in  accordance  with  the 
DOT&E-approved  IMTP. 

•  The  BMDS  Operational  Test  Agency  and  the  MDA  attempted 
FTO-02  Event  1  in  June  2015  at  the  Pacific  Missile  Range 
Facility  on  Kauai,  Hawaii.  The  MDA  intended  to  demonstrate 
the  operational  capability  of  the  regionaPtheater  European 
Phased,  Adaptive  Approach  Phase  2  BMDS,  anchored  by  the 
AAMDS,  to  defend  Europe  against  medium-range  ballistic 
missiles.  The  test  was  to  be  the  first  target  intercept  by  the 
AAMDS  and  the  first  flight  for  the  SM-3  Block  IB  with 
threat  update  guided  missile.  Due  to  a  target  malfunction, 

the  test  was  not  completed  and  the  SM-3  guided  missile  was 
not  launched.  This  test  is  scheduled  to  be  attempted  again  in 
December  2015. 

•  The  BMDS  Operational  Test  Agency  and  the  MDA  conducted 
FTO-02  Event  2  and  Event  2a  in  September  and  October  2015 
at  Wake  Island  and  the  broad-ocean  area  surrounding  it. 

The  Operational  Test  Agency  designed  the  test  mission  to 
demonstrate  a  layered  BMDS  with  multiple  combat  systems 


sharing  common  defended  areas  and  shot  opportunities  against 
two  threat-representative  ballistic  missiles.  The  primary 
test  objective  was  to  assess  Aegis  BMD  system  capability  to 
prosecute  a  ballistic  missile  threat  engagement  in  the  presence 
of  non-organic  post-intercept  debris,  while  simultaneously 
conducting  Anti-Air  Warfare.  The  THAAD  combat  system, 
using  Lot  4  interceptors  for  the  first  time,  generated  the 
non-organic  post-intercept  debris  scene  for  Aegis  BMD. 

•  The  MDA  did  not  conduct  Homeland  Defense  flight  testing  in 
FY15. 

•  During  FY15,  the  MDA  conducted  four  major  ground  tests. 

-  Ground  Test,  Distributed-04e  (GTD-04e)  Part  2  and 
Ground  Test,  Integrated-06  (GTI-06)  Part  3  in  January  and 
July  2015.  These  tests  focused  on  USNORTHCOM  and 
USPACOM  scenarios,  respectively. 

-  GTI-06  Part  1  and  GTD-06  Part  1  in  May  and 
October  2015.  These  tests  focused  on  USEUCOM  and 
USCENTCOM  scenarios,  respectively. 
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•  The  MDA  also  conducted  several  wargames  and  exercises 
designed  to  enhance  Combatant  Command  BMD  readiness 
and  increase  Service  member  confidence  in  the  deployed 
elements  of  the  BMDS. 

Assessment 

•  The  MDA,  in  collaboration  with  DOT&E,  updated  the 
IMTP  to  incorporate  BMDS  element  maturity,  program 
modifications,  and  fiscal  constraints.  The  MDA  included  in 
the  IMTP  a  much  closer  tie  between  individual  tests  and  the 
planned  BMDS  technical  capability  increment  deliveries. 

•  FTO-02  Event  2  and  Event  2a  demonstrated  that  the  BMDS 
capability  against  theater/regional  threats  increased  during 
FY15.  THAAD  interceptors  hit  one  short-range  and  one 
medium-range  threat-representative  ballistic  missile  targets 
while  Aegis  BMD  simultaneously  engaged  an  air-breathing 
threat  with  SM-2  Block  IIIA  guided  missiles.  However,  the 
SM-3  Block  IB  Threat  Update  guided  missile  also  targeting  the 
medium-range  ballistic  missile  target,  failed  early  in  flight;  an 
engineering  Failure  Review  Board  investigation  is  underway. 
Full  assessment  of  the  FTO-02  Event  2  and  Event  2a  test 
mission  data  with  respect  to  BMDS  operational  effectiveness, 
operational  suitability,  and  interoperability  is  ongoing. 

•  With  no  flight  testing  in  FY 15,  the  Homeland  Defense 
assessment  remains  unchanged.  The  GMD  combat  system 
demonstrates  a  limited  capability  to  defend  the  U.S.  Homeland 
from  small  numbers  of  intermediate-range  or  intercontinental 
ballistic  missile  threats  launched  from  North  Korea  or  Iran. 

•  Ground  testing  was  able  to  demonstrate  interoperability 
and  some  command  and  control  capabilities  for 
Combatant  Command  architectures. 

-  C2BMC  tasked  and  managed  the  two  USPACOM 

AN/TPY-2  (FBM)  radars  for  the  first  time  and  exercised 


cross-area  of  responsibility  data  sharing  between 
USEUCOM  and  USCENTCOM. 

-  Ground  tests  also  identified  several  tasking  problems 
between  C2BMC  and  AN/TPY-2  (FBM),  which  are  under 
evaluation. 

-  Many  of  the  models  and  simulations  used  in  the 
ground  tests  have  no  accreditation,  which  limits  the 
MDA’s  capability  to  produce  BMDS-level  performance 
assessments.  The  MDA  is  developing  a  high-fidelity, 
end-to-end,  digital  performance  assessment  modeling  and 
simulation  capability  for  the  BMDS;  this  effort  will  take 
several  more  years. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  has 

addressed  most  previous  BMDS  recommendations.  The 

following  recommendations  from  FY14  remain  outstanding: 

1 .  The  MDA  should  continue  to  address  recommendations 
made  in  the  DOT&E  FTO-Ol  assessment  found  in  the 
classified  DOT&E  February  2014  BMDS  Annual  Report, 
Appendix  E. 

2.  The  MDA  should  increase  the  development  priority 
and  associated  funding  for  the  BMDS  simulation-based 
performance  assessment  capability.  The  ability  to  produce 
high-fidelity  and  statistically  significant  BMDS-level 
performance  assessments  is  critical. 

•  FY15  Recommendation. 

1.  The  MDA  should  include  Patriot  in  system-level  operational 
flight  test  events  in  order  to  assess  interoperability  and 
integration  between  all  of  the  BMDS  combat  systems  and 
sensors. 
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Aegis  Ballistic  Missile  Defense  (Aegis  BMD) 


Executive  Summary 

•  The  Missile  Defense  Agency  (MDA)  conducted  eight  Aegis 
Ballistic  Missile  Defense  (Aegis  BMD)  flight  tests  in  FY15 
and  two  in  1QFY16.  All  but  one  intercept  attempt  resulted 
in  successful  intercepts.  Overall.  Aegis  BMD  successfully 
engaged  four  ballistic  missile  targets  and  five  anti-air  warfare 
targets. 

•  In  FY15,  the  Aegis  BMD  program  conducted  combined 
developmental  testing/operational  testing  (DT/OT)  and 
operational  flight  testing  of  the  Aegis  Baseline  9  system 
in  its  Aegis  Ashore  (Baseline  9.B1)  and  Aegis  Afloat 
(Baseline  9. Cl)  configurations  with  Standard  Missile-3 
(SM-3)  Block  IB,  SM-6  Dual  I,  and  SM-2  Block  IV  guided 
missiles,  and  also  conducted  FOT&E  of  the  Aegis  BMD 
4.0  system.  The  program  continued  conducting  early 
developmental  flight  testing  of  the  SM-3  Block  IIA-guided 
missile  in  accordance  with  the  DOT&E  approved  Integrated 
Master  Test  Plan.  The  Aegis  Ashore  configuration  supports 
the  President’s  European  Phased- Adaptive  Approach  (EPAA) 
for  the  defense  of  Europe. 

•  Although  the  program  completed  FOT&E  for  the  Aegis  BMD 
3.6.1  system  in  FY11,  the  program  continued  to  use  a  variant 
of  the  Aegis  BMD  3.6  system  (i.e.,  3.6.3)  in  system-level  tests 
from  January  2015  through  October  2015,  and  a  fleet  exercise 
in  October  2015  to  assess  system-level  engagement  capability 
and  interoperability  with  the  Ballistic  Missile  Defense  System 
(BMDS)  and  foreign  missile  defense  assets. 

•  Operational  Aegis  BMD  assets  and  hardware-in-the-loop 
(HW1L)  facilities  underwent  cybersecurity  and  reliability, 
maintainability,  and  availability  testing,  and  participated  in 
several  flight  and  ground  tests  to  assess  Aegis  Ashore  and 
Afloat,  BMD  4.0.2,  BMD  4.0.3,  and  BMD  3.6.3  capabilities 
and  interoperability  with  the  BMDS. 

•  Testing  demonstrated  engagement  capabilities  against 
short-range  ballistic  missiles  in  both  endo-  and 
exo-atmospheric  engagements  and  against  anti-air  warfare 
targets. 

•  During  an  Aegis  BMD  4.0  test,  the  system  demonstrated 
its  Distributed  Weighted  Engagement  Scheme  (DWES),  an 
automated  engagement  coordination  capability  involving 
multiple  firing  ships. 

•  Two  third-stage  rocket  motor  (TSRM)  failures  in  FY11  and 
FY13  lower  certainty  in  SM-3  Block  IB  missile  reliability 
in  its  currently  fielded  configuration.  Following  failure 
investigations,  the  MDA  made  a  software  change  and  a 
hardware  redesign  of  the  TSRM  aft  nozzle.  Ground  testing 
of  the  redesigned  nozzle  began  in  FY14  and  flight  testing  is 
expected  in  February  2016. 

•  A  TSRM  cold  gas  regulator  (CGR)  anomaly  observed  during 
testing  is  under  investigation  by  the  MDA. 


Aegis  Cruiser  Aegis  Ashore  and  Vertical  Launch  System 


•  Flight  testing,  modeling  and  simulation,  and  ground  testing 
have  demonstrated  Aegis  BMD  4.0  and  Aegis  Afloat 
capabilities  to  perform  Long-Range  Surveillance  and  Track 
(LRS&T). 

System 

•  Aegis  BMD  is  a  sea-based  missile  defense  system  that 
employs  the  multi-mission  shipboard  Aegis  Weapon  System, 
with  improved  radar  and  new  missile  capabilities  to  engage 
ballistic  missile  threats.  Capabilities  of  Aegis  BMD  include: 

-  Computer  program  modifications  to  the  AN/SPY-1  radar 
for  LRS&T  of  ballistic  missiles  of  all  ranges 

-  A  modified  Aegis  Vertical  Launching  System,  which  stores 
and  fires  SM-3  Block  IA  and  Block  IB  guided  missiles, 
modified  SM-2  Block  IV  guided  missiles,  and  SM-6  Dual  I 
guided  missiles 

-  SM-3  Block  IA  and  Block  IB  guided  missiles  that  use 
maneuverable  kinetic  warheads  to  accomplish  midcourse 
engagements  of  short-,  medium-,  and  intermediate-range 
ballistic  missiles 

-  Modified  SM-2  Block  IV  guided  missiles  that  provide 
terminal  engagement  capability  against  short-range 
ballistic  missiles 

-  SM-6  Dual  I  guided  missiles  that  provide  terminal 
engagement  capability  against  short-range  ballistic  missiles 

•  Aegis  Ashore  (Baseline  9.B1)  is  a  land-based  version  of 
Aegis  BMD,  with  an  AN/SPY- 1  radar  and  Vertical  Launching 
System  to  enable  engagements  against  medium-  and 
intermediate-range  ballistic  missiles  with  SM-3  guided 
missiles.  Once  the  MDA  deploys  Aegis  Ashore  to  Romania,  it 
will  become  the  central,  land-based  component  of  the  second 
phase  of  the  European  Phased-Adaptive  Approach  for  the 
defense  of  Europe. 

•  Aegis  BMD  and  Aegis  Ashore  are  capable  of  performing 
autonomous  missile  defense  operations  and  operations  that 
exploit  networked  sensor  information  by  sending/receiving 
cues  to/from  other  BMDS  sensors  through  tactical  datalinks. 
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Mission 

The  Navy  can  accomplish  three  missile  defense-related  missions 

using  Aegis  BMD: 

•  Defend  deployed  forces  and  allies  from  short-  to 
intermediate-range  theater  ballistic  missile  threats 

•  Provide  forward-deployed  radar  capabilities  to  enhance 
defense  against  ballistic  missile  threats  of  all  ranges  by 
sending  cues  or  target  track  data  to  other  elements  of  the 
BMDS 

•  Provide  all  short-  to  long-range  ballistic  missile  threat  data 
to  the  Command  and  Control,  Battle  Management,  and 
Communications  (C2BMC)  system  for  dissemination  to 


Combatant  Commanders’  headquarters  to  ensure  situational 
awareness 

Major  Contractors 

•  Aegis  BMD  Weapon  System:  Lockheed  Martin  Corporation, 
Mission  Systems  and  Training  -  Moorestown,  New  Jersey 

•  AN/SPY- 1  Radar:  Lockheed  Martin  Corporation,  Mission 
Systems  and  Training  -  Moorestown,  New  Jersey 

•  SM-3  and  SM-6  Missile:  Raytheon  Company,  Missile 
Systems  -  Tucson,  Arizona 


Activity 

•  From  November  2014  through  October  2015,  the  MDA 
conducted  combined  DT/OT  and  operational  flight  testing 
of  the  Aegis  Baseline  9  system  in  its  Aegis  Ashore  and 
Afloat  configurations  with  SM-3  Block  IB  guided  missiles, 
and  conducted  FOT&E  of  the  Aegis  BMD  4.0  system. 
Additionally,  in  June  2015,  the  MDA  conducted  early 
developmental  flight  testing  of  the  SM-3  Block  IIA  guided 
missile. 

•  Although  the  program  completed  FOT&E  for  the  Aegis  BMD 
3.6.1  system  in  FY11,  the  program  continued  to  use  a  variant 
of  the  Aegis  BMD  3.6  system  (i.e.,  3.6.3)  in  system-level 
tests  and  a  fleet  exercise  in  1QFY16  to  assess  system-level 
engagement  capability  and  interoperability  with  the  BMDS 
and  foreign  missile  defense  assets. 

•  The  Aegis  BMD  program  planned  to  conduct  nine  flight 
tests  in  FY15,  but  only  completed  eight.  The  program 
also  conducted  two  intercept  missions  in  1QFY16.  Aegis 
BMD  successfully  engaged  four  ballistic  missile  targets  and 
five  anti-air  warfare  targets. 

-  In  November  2014,  the  MDA  conducted  Flight  Test 
Standard  Missile-25  (FTM-25),  an  Integrated  Air  and 
Missile  Defense  mission  where  an  Aegis  BMD  5.0  with 
Capability  Upgrade  destroyer,  operating  in  Integrated  Air 
and  Missile  Defense  priority  mode,  detected,  tracked,  and 
performed  an  SM-3  Block  IB  intercept  of  a  separating 
short-range  ballistic  missile,  while  simultaneously 
conducting  an  anti-air  warfare  raid  engagement  against  a 
raid  of  two  subsonic  cruise  missile  surrogates. 

-  The  BMDS  Operational  Test  Agency  and  the  MDA 
attempted  Flight  Test  Operational-02  (FTO-02)  Event  1  in 
June  2015,  at  the  Pacific  Missile  Range  Facility  on  Kauai, 
Hawaii.  The  BMDS  Operational  Test  Agency  and  MDA 
intended  to  demonstrate  the  operational  capability  of  the 
regional/theater  EPAA  Phase  2  BMDS,  anchored  by  the 
Aegis  Ashore  combat  system,  to  defend  Europe  against 
medium-range  ballistic  missiles.  The  test  was  intended 

to  be  the  first  target  intercept  by  the  Aegis  Ashore  combat 
system  and  the  first  flight  test  of  the  SM-3  Block  IB  Threat 
Update  guided  missile.  Due  to  a  target  malfunction,  the 


test  was  not  completed  and  the  SM-3  guided  missile  was 
not  launched.  This  test  is  scheduled  to  be  attempted  again 
in  December  2015. 

-  In  a  four-event  Multi-Mission  Warfare  (MMW)  test 
campaign  in  July  2015,  an  Aegis  BMD  5.0  with  Capability 
Upgrade  destroyer  detected,  tracked,  and  engaged  cruise 
missile  and  short-range  ballistic  missile  targets.  In  MMW 
Event  1,  the  ship  intercepted  a  short-range  ballistic  missile 
in  the  endo-atmosphere  with  an  SM-6  Dual  I  missile.  This 
was  the  first  intercept  of  a  ballistic  missile  using  an  SM-6 
Dual  1  using  the  Baseline  9  sea-based  terminal  capability. 

In  MMW  Event  2,  the  ship  intercepted  a  short-range 
ballistic  missile  target  with  an  SM-2  Block  IV  missile, 
demonstrating  retention  of  Aegis  BMD  3.6.1  near-term 
sea-based  terminal  capability.  Events  3  and  4  of  the  MMW 
campaign  included  SM-6  Dual  1  missile  engagements  of 
anti-ship  cruise  missile  surrogates  to  demonstrate  retention 
of  air  defense  capability  with  the  SM-6  Dual  1  missile. 

-  At-Sea  Demonstration- 15  in  October  2015  was  a 
multi-event  fleet  exercise  wherein  assets  from  North 
Atlantic  Treaty  Organization  member  countries 
explored  the  exchange  of  air  and  ballistic  missile 
message  information  across  operational  communication 
architectures  during  cruise  missile  and  ballistic  missile 
engagements.  One  of  the  nine  events  in  the  exercise 
included  an  engagement  of  a  short-range  non-separating 
ballistic  missile  by  an  Aegis  BMD  3.6.3  destroyer 
with  an  SM-3  Block  IA  guided  missile.  In  the  live  fire 
engagement,  the  ship  detected,  tracked,  and  intercepted  the 
ballistic  missile  target.  Participating  assets  also  included 
an  Aegis  BMD  3.6.3  laboratory  representation,  an  Aegis 
5.3.10  air  defense  ship,  C2BMC,  and  allied  naval  vessels 
from  Great  Britain,  Spain,  Netherlands,  Italy,  Canada, 
France,  and  Norway. 

-  The  BMDS  Operational  Test  Agency  and  the  MDA 
conducted  FTO-02  Event  2  and  Event  2a  in  September  and 
October  2015  at  Wake  Island  and  the  broad-ocean  area 
surrounding  it.  The  MDA  designed  the  test  mission 

to  demonstrate  a  layered  BMDS  with  multiple  combat 
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systems  sharing  common  defended  areas  and  shot 
opportunities  against  two  threat-representative  ballistic 
missiles.  The  primary  test  objective  was  to  assess 
Aegis  BMD  system  capability  to  prosecute  a  ballistic 
missile  threat  engagement  in  the  presence  of  non-organic 
post-intercept  debris,  while  simultaneously  conducting 
anti-air  warfare  against  an  air-breathing  target.  The 
Tenninal  High-Altitude  Area  Defense  (THAAD)  combat 
system  generated  the  non-organic  post-intercept  debris 
scene  for  Aegis  BMD. 

•  Aegis  BMD  participated  in  other  live-target  or  live-interceptor 
test  events  in  FY15. 

-  In  Flight  Test  Other-20  (FTX-20)  in  October  2014,  the 
MDA  used  an  Aegis  BMD  5.0  with  Capability  Upgrade 
destroyer  to  conduct  a  simulated  SM-3  Block  IB 
engagement  of  a  separating  medium-range  ballistic 
missile.  Although  no  live  or  simulated  missiles  were  fired 
at  the  target,  the  system  did  exercise  several  fire  control, 
discrimination,  and  engagement  functions. 

-  In  FTX-19  in  February  2015,  two  Aegis  BMD  4.0.2 
destroyers  detected,  tracked,  and  conducted  simulated 
SM-3  Block  IB  engagements  against  three  short-range 
separating  ballistic  missile  targets  in  a  raid  scenario.  This 
was  the  first  simulated  engagement  of  a  raid  of  three 
separating  targets  with  the  Aegis  BMD  4.0  system  and 
the  first  live-target  mission  involving  Aegis  BMD  DWES, 
which  provides  automated  engagement  coordination 
between  multiple  firing  ships.  Additionally,  an  Aegis  BMD 
5.0  with  Capability  Upgrade  destroyer  participated  in  the 
test  to  explore  the  capability  of  that  system  to  perform 

air  and  ballistic  missile  defense  against  a  raid  of  ballistic 
missiles  and  a  simulated  raid  of  supersonic  sea-skimming 
anti-ship  cruise  missiles  using  simulated  guided  missiles 
while  operating  in  Integrated  Air  and  Missile  Defense 
priority  mode  with  remote  engagement  authorized. 

-  In  June  2015,  the  SM-3  Block  IIA  Cooperative 
Development  Control  Test  Vehicle-01  (SCD  CTV-01)  flight 
test  demonstrated  SM-3  Block  IIA  flight  through  nosecone 
deployment  and  TSRM  flight.  This  was  the  first  live  fire 
event  for  the  SM-3  Block  IIA  guided  missile,  which  is  a 
joint  U.S. -Japanese  development  of  a  21-inch  diameter 
variant  of  the  SM-3.  This  test  was  the  third  in  a  series  of 
six  test  events  to  validate  missile  and  canister  designs  for 
the  Block  IIA  guided  missile. 

•  In  FY15,  operational  Aegis  BMD  assets  and  HW1L  facilities 
underwent  cybersecurity  and  reliability,  maintainability,  and 
availability  testing,  and  participated  in  several  flight  and 
ground  tests  to  assess  Aegis  Ashore  and  Afloat,  BMD  4.0, 
and/or  BMD  3.6.3  capabilities  and  interoperability  with  the 
BMDS. 

-  In  January  2015,  Ground  Test  Distributed-04e  (GTD-04e) 
Part  2,  the  MDA  tested  the  engagement  and  sensor 
capabilities  of  fielded  and  to-be-fielded  missile  defense 
elements  and  sensors  against  ballistic  missiles  of  all  ranges 
in  a  distributed  environment.  Participants  included  Aegis 
BMD  4.0.2  and  3.6.3  (laboratory  site  with  sailors  from 


the  US S  Lake  Erie,  USS  Decatur,  and  USS  Hopper  on 
console),  C2BMC,  Patriot,  THAAD,  Space-Based  Infrared 
System  (SB1RS),  AN/TPY-2  (Forward-Based  Mode 
(FBM)),  Ground-Based  Midcourse  Defense  (GMD),  and 
Upgraded  Early  Warning  Radars. 

-  During  Ground  Test  Integrated-06  (GTI-06)  Part  1  in 
April  and  May  2015,  the  MDA  tested  the  engagement  and 
sensor  capabilities  of  fielded  and  to-be-fielded  missile 
defense  elements  and  sensors  against  ballistic  missiles  of 
all  ranges  in  an  HW1L  environment.  A  key  focus  of  the 
test  was  the  demonstration  of  the  new  Aegis  Ashore  and 
Afloat  capabilities  within  a  realistic  BMDS  architecture, 
including  engagement  coordination  with  and  without  the 
Aegis  DWES  capability.  Participants  included  HW1L 
representations  of  Aegis  Ashore  and  Afloat  assets,  Aegis 
BMD  4.0.2  and  3.6.3  (laboratory  site  with  sailors  from 
the  Aegis  Ashore  Missile  Defense  Facility  in  Romania, 

USS  Arleigh  Burke,  and  USS  Lake  Erie  on  console), 
C2BMC,  Patriot,  THAAD,  SBIRS,  and  AN/TPY-2  (FBM). 

-  In  GTI-06  Part  3  in  July  2015,  the  MDA  utilized  a  test 
architecture  with  HWIL  representations  to  evaluate 
interoperability  between  Aegis  BMD  variants  and  GMD, 
and  interoperability  between  GMD  and  other  strategic 
assets.  Assets  included  Aegis  BMD  4.0.3,  Aegis  Afloat, 
C2BMC,  SBIRS,  AN/TPY-2  (FBM),  and  the  Sea-Based 
X-band  Radar  that  was  represented  but  not  operational. 

-  The  Commander,  Operational  Test  and  Evaluation  Force 
conducted  a  cybersecurity  Cooperative  Vulnerability  and 
Penetration  Assessment  of  Aegis  Ashore  in  August  2015 
at  the  Aegis  Ashore  Missile  Defense  Facility  in  Romania. 
The  MDA  also  planned  to  conduct  an  Adversarial 
Assessment  following  the  Cooperative  Vulnerability  and 
Penetration  Assessment;  however,  the  MDA  postponed 
this  testing  due  to  incomplete  construction  and  system 
integration  at  the  Aegis  Ashore  Site. 

-  The  Aegis  Ashore  Missile  Defense  Test  Complex  at 
Pacific  Missile  Range  Facility  underwent  a  maintenance 
demonstration  in  September  2015. 

-  In  GTD-06  Part  1  in  October  2015  utilized  a  distributed 
test  environment  to  demonstrate  Aegis  Ashore  in  Romania, 
Aegis  Afloat,  and  AN/TPY-2  (FBM)  capabilities  within 

a  realistic  BMDS  architecture.  Other  BMDS  assets 
supporting  the  test  included  C2BMC,  SBIRS,  Upgraded 
Early  Warning  Radar,  Patriot,  THAAD,  and  laboratory 
representations  of  Aegis  BMD  4.0.3  and  Aegis  BMD  3.6.3. 

Assessment 

•  In  F Y 1 5  and  1 QF Y 16,  the  Aegis  Afloat  system  underwent 
DT/OT  and  operational  flight  testing  of  that  system’s 
exo-atmospheric  engagement  capabilities  (during  FTX-20, 
FTM-25,  and  FTO-02  Event  2a)  and  its  endo-atmospheric 
engagement  capabilities  with  SM-6  Dual  I  and  SM-2  Block  IV 
missiles  (during  MMW  Events  1  and  2).  Testing  demonstrated 
engagement  capabilities  against  short-range  ballistic  missiles 
in  both  exo-  and  endo-atmospheric  engagements.  Additional 
flight  testing  and  high-fidelity  modeling  and  simulation 
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analyses  are  needed  to  quantitatively  evaluate  the  effectiveness 
of  the  Baseline  9  system  at  engaging  ballistic  missiles  in  the 
exo-  and  endo-atmospheric  phases  of  flight  for  a  range  of 
scenarios. 

•  FTM-25  demonstrated  the  use  of  Integrated  Air  and  Missile 
Defense  radar  priority  mode  in  a  live  engagement  during 
which  cruise  missile  and  ballistic  missile  targets  were 
simultaneously  engaged,  although  only  for  a  less-than-fully 
stressing  case,  with  a  single  ballistic  missile  and  a  raid  of 
two  subsonic  cruise  missile  targets. 

•  Three  of  the  MMW  events  (Events  1,  3,  and  4)  demonstrated 
that  SM-6  Dual  I  missiles  can  be  used  to  conduct  sea-based 
terminal  engagements  against  short-range  non-separating 
ballistic  missiles,  and  that  they  retain  the  air  defense 
capabilities  that  were  demonstrated  during  SM-6  IOT&E  and 
FOT&E  flight  testing. 

•  The  MDA  intended  FTO-02  Event  2  and  Event  2a  to 
demonstrate  the  Aegis  Afloat  capability  to  prosecute  a  ballistic 
missile  threat  engagement  as  part  of  a  layered  BMDS  in 

the  presence  of  non-organic  post  intercept  debris,  while 
simultaneously  conducting  an  anti-air  warfare  engagement  in 
Integrated  Air  and  Missile  Defense  priority  mode.  However, 
shortly  following  launch,  the  SM-3  Block  IB  Threat  Update 
guided  missile  targeting  the  medium-range  ballistic  missile 
target  failed.  Prior  to  this,  a  THAAD  interceptor  intercepted 
the  short-range  ballistic  missile  target,  generating  debris  that 
may  enable  accurate  modeling  and  simulation  of  Aegis  BMD 
combat  system  capability  in  the  presence  of  post  intercept 
debris.  At  the  same  time  Aegis  BMD  was  attempting  to 
engage  the  ballistic  missile  target  with  the  SM-3  missile,  it 
succeeded  in  engaging  an  air-breathing  target  with  two  SM-2 
Block  111A  guided  missiles.  An  engineering  Failure  Review 
Board  investigation  is  underway  to  determine  the  root  cause 
of  the  SM-3  guided  missile  failure.  A  full  assessment  of  the 
FTO-02  Event  2  and  Event  2a  test  mission  data  with  respect  to 
Aegis  BMD  and  BMDS  operational  effectiveness,  operational 
suitability,  and  interoperability  is  ongoing. 

•  The  Aegis  BMD  4.0  system,  which  is  the  latest  deployed 
version  of  Aegis  BMD  and  is  the  primary  sea-based  firing 
asset  for  EPAA  Phase  2,  conducted  follow-on  testing  in 
FY15  to  supplement  the  IOT&E  flight  testing  and  modeling 
and  simulation  conducted  in  FY13  and  FY14.  The  most 
significant  capability  demonstrated  was  the  BMD  4.0  system’s 
DWES,  an  automated  engagement  coordination  capability, 
during  the  FTX-19  mission.  In  that  mission,  two  Aegis 
BMD  4.0  ships  demonstrated  that  the  DWES  capability  can 
determine  the  preferred  shooter  for  a  given  ballistic  missile 
engagement  when  two  Aegis  BMD  firing  assets  are  present, 
thereby  reducing  missile  wastage  while  ensuring  BMD  threat 
coverage. 

•  Prior  IOT&E  flight  testing  and  supporting  modeling 
and  simulation  demonstrated  that  Aegis  BMD  4.0  has 
the  capability  to  engage  and  intercept  non-separating, 
simple-separating,  and  complex-separating  ballistic  missiles 
in  the  midcourse  phase  with  SM-3  Block  IB  guided  missiles. 
However,  flight  testing  and  modeling  and  simulation  are  not 
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yet  sufficient  to  assess  the  full  range  of  expected  threat  types, 
threat  ground  ranges,  and  threat  raid  sizes.  Details  on  the 
BMD  4.0  system’s  performance  can  be  found  in  the  classified 
December  2014  Aegis  BMD  4.0  IOT&E  report. 

•  Reliability,  maintainability,  and  availability  data  collected 
during  Baseline  9  BMD-related  testing  in  early  to  mid  FY15 
shows  that  the  Baseline  9  system  does  not  currently  meet 
its  requirements  for  availability  and  the  mean  time  to  repair 
hardware,  mostly  due  to  a  series  of  early  Aegis  Display  System 
failures  and  an  AN/SPY- 1  radar  coolant  leak  that  downed  the 
system  for  an  extended  period  of  time.  The  majority  of  the 
Aegis  Display  System  problems  have  been  resolved  by  the 
installation  of  new  graphics  cards  for  each  console.  Additional 
data  collected  during  late  FY15  to  early  FY16  are  under 
review  by  data  scoring  boards.  It  is  uncertain  at  present  if 
additional  data  collection  periods  are  needed  to  prove  that  the 
system’s  suitability  is  sufficient  for  operational  use. 

•  The  limited  number  of  SM-3  Block  IB  firings  (10  as  of 
FTO-02  Event  2a)  and  the  2  TSRM  failures  (during  FTM-16 
Event  2  in  FY11  and  FTM-21  in  FY13)  lower  certainty  in 
overall  SM-3  Block  IB  missile  reliability  in  its  currently 
fielded  configuration.  The  program  addressed  and  tested  a 
correction  for  the  first  of  the  SM-3  TSRM  failures  when  it 
modified  the  TSRM’s  inter-pulse  delay  time  between  axial 
thrust  burns.  This  correction,  which  the  MDA  implemented 
following  the  FTM-16  Event  2  failure,  did  not  prevent  the 
TSRM  failure  in  the  second  of  two  salvo-launched  SM-3 
Block  IB  guided  missiles  in  FTM-21.  The  MDA  established 
a  Failure  Review  Board  (FRB)  to  determine  the  root  cause 
of  this  failure  and  the  FRB  uncovered  enough  evidence  to 
determine  that  a  redesign  was  needed  for  the  TSRM  aft  nozzle. 
Ground  testing  of  the  new  design  began  in  FY14.  Flight 
testing  of  the  new  design  is  expected  in  February  2016.  The 
new  nozzle  design  can  be  retrofitted  into  current  SM-3  Block 
1A  and  Block  IB  missiles. 

•  The  FTM-25  flight  test  and  recent  lot  acceptance  testing 
have  shown  that  the  TSRM  Attitude  Control  System  CGR, 
which  the  MDA  re-designed  following  FTM-15,  can  produce 
anomalous  low  regulated  pressure  levels.  In  five  flight 
tests  following  FTM-15,  the  TSRM  showed  no  anomalous 
behavior.  The  CGR  anomaly  in  FTM-25  did  not  preclude  a 
successful  intercept;  however,  the  cold  gas  pressure  observed 
was  much  lower  than  that  commanded.  If  the  regulated 
pressure  from  the  CGR  is  too  low,  the  Attitude  Control 
System  may  not  function  properly.  Analysis  suggests  that 
now  defunct  tooling  procedures  caused  the  FTM-25  CGR 
anomaly.  The  manufacturer  built  the  CGR  flown  in  FTM-25 
using  old  tooling  procedures  (it  was  the  second  CGR  built 
following  the  re-design  after  FTM-15).  The  MDA  established 
an  industry-led  FRB  to  determine  the  root  cause  of  the  low 
pressure  outputs  from  the  CGRs,  and  its  investigation  is 
ongoing.  The  CGR  anomaly  is  not  related  to  the  TSRM 
inter-pulse  delay  problem  or  the  aft  nozzle  deficiency 
previously  discussed. 

•  Flight  testing,  modeling  and  simulation,  and  ground  testing 
have  demonstrated  the  Aegis  BMD  4.0  capability  to  perform 
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the  LRS&T  mission.  The  Flight  Test  Ground-Based 
lnterceptor-07  (FTG-07)  mission  in  FY13  highlighted  the 
need  to  further  explore  and  refine  tactics,  techniques,  and 
procedures  (TTPs)  for  the  transmission  and  receipt  of  Aegis 
BMD  track  data  for  GMD  use.  The  MDA  demonstrated  in 
GTI-06  Part  3  the  Aegis  BMD  4.0  software’s  ability  to  provide 
track  data  that  GMD  can  use.  The  MDA  will  test  Aegis  Afloat 
systems  in  a  future  ground  test. 

•  All  components  of  the  SM-3  Block  IIA  guided  missile  flight 
tested  thus  far  during  developmental  testing  have  performed 
as  designed.  SCD  CTV-01  in  FY15  showed  good  missile 
performance  from  egress  from  the  Vertical  Launching  System, 
to  Stage  1  burn,  to  Stage  1/2  separation,  to  Second  Stage 
Rocket  Motor  burn,  to  Stage  2/3  separation,  to  nosecone 
jettison,  and  to  TSRM  burn. 

•  At-Sea  Demonstration- 15  demonstrated  that  Aegis  BMD  can 
interoperate  with  North  Atlantic  Treaty  Organization  defenses, 
and  exchange  air  and  ballistic  missile  message  information 
across  operational  communication  architectures  during  cruise 
missile  and  ballistic  missile  engagements.  In  the  live  fire  test, 
the  Aegis  BMD  3.6.3  ship  detected,  tracked,  and  intercepted 

a  short-range  non-separating  ballistic  missile  target  using  an 
SM-3  Block  IA  guided  missile. 

•  Cybersecurity  testing  results  will  be  included  in  the  classified 
2015  BMDS  Annual  Report. 

•  The  MDA  continues  to  utilize  Aegis  BMD  assets  and  F1W1L 
representations  in  ground  test  events,  which  has  helped  to 
refine  TTPs  and  overall  interoperability  of  the  system  with  the 
BMDS.  However,  the  test  events  routinely  demonstrated  that 
inter-element  coordination  and  interoperability  are  still  in  need 
of  improvement. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  program: 

1.  Partially  addressed  the  first  recommendation  from  FY13 
to  conduct  flight  testing  of  the  Aegis  BMD  4.0  remote 
engagement  authorized  capability  against  a  medium-  or 
intermediate-range  ballistic  missile  target  using  an  SM-3 
Block  IB  guided  missile,  when  it  conducted  FTO-02  Event 
2a.  This  assumes  that  DOT&E  can  use  modeling  and 
simulation  results  to  determine  if  the  Aegis  combat  system 
successfully  supported  the  engagement.  Although  the  MDA 
conducted  FTO-02  Event  2a  with  an  Aegis  BMD  5.0  with 
Capability  Upgrade  destroyer,  rather  than  a  BMD  4.0  ship, 
the  Aegis  BMD  4.0  and  Aegis  Afloat  remote  engagement 
capabilities  are  similar.  Due  to  the  SM-3  guided  missile 
failure  during  FTO-02  Event  2a,  the  MDA  should  plan  to 
conduct  an  end-to-end  remote  engagement  authorized  flight 
test  using  track  data  from  a  forward-based  sensor. 

2.  Partially  addressed  the  second  recommendation  from  FY13 
to  conduct  operationally  realistic  testing  that  exercises 
Aegis  BMD  4.0’s  improved  engagement  coordination  with 
THAAD  and  Patriot  when  it  conducted  FTO-02  Event  2a 
using  Aegis  Afloat  and  THAAD  firing  assets.  The  flight 
test  did  not  include  a  Patriot  firing  asset,  so  engagement 
coordination  with  Patriot  has  not  been  flight  tested  to  date. 


3.  Addressed  the  fourth  recommendation  from  FY13  to  use  the 
FRB  process  to  identify  the  failure  mechanism  responsible 
for  the  FTM-21  second  missile  failure  and  determine  the 
underlying  root  cause  that  may  be  common  to  both  the 
FTM-16  Event  2  and  FTM-21  second  missile  failures 

by  completing  the  FRB  process  for  the  TSRM  failures 
encountered  to  date.  The  MDA  plans  to  flight-test  the 
redesigned  aft  nozzle  area  of  the  TSRM  in  February  2016. 

4.  Addressed  the  fifth  recommendation  from  FY13  to  deliver 
sufficient  Aegis  BMD  4.0  validation  data  and  evidence 

to  support  BMDS  modeling  and  simulation  verification, 
validation,  and  accreditation  (VV&A)  of  the  Aegis 
HWIL  and  digital  models.  The  program  did  so  when 
the  Commander,  Operational  Test  and  Evaluation  Force 
provided  VV&A  evidence  for  the  digital  models  used 
for  element-level  performance  analyses  in  support  of  the 
operational  assessment  of  the  Aegis  BMD  4.0  system  with 
SM-3  Block  IB  guided  missiles. 

5.  Addressed  the  first  recommendation  from  FY14  to  conduct 
flight  tests  or  high-fidelity  modeling  and  simulation 
analyses  to  demonstrate  the  Aegis  BMD  4.0  system’s 
capability  to  perform  LRS&T  of  a  raid  of  long-range 
threats.  The  Aegis  BMD  4.0.3  update  improves  the  LRS&T 
of  long-range  threats  and  the  MDA  tested  this  capability  in 
GTI-06  Part  3  for  various  raid  sizes. 

6.  Has  partially  addressed  the  second  recommendation  from 
FY14  to  determine  the  appropriate  LRS&T  TTPs  for  the 
transmission  and  receipt  of  Aegis  BMD  4.0  track  data  for 
GMD  use.  The  MDA  added  GTI-06  Part  3  to  the  Integrated 
Master  Test  Plan  to  demonstrate  that  GMD  can  use  data 
provided  by  Aegis  BMD  4.0.3,  which  has  improved  LRS&T 
capability,  when  the  data  are  transmitted  as  per  design. 

7.  Has  partially  addressed  the  third  recommendation  from 
FY 14  to  ensure  that  sufficient  flight  testing  of  the  Aegis 
Afloat  system  is  conducted  to  allow  for  VV&A  of  the 
modeling  and  simulation  suite  to  cover  the  full  design  to 
Aegis  BMD  battlespace  of  threat  ballistic  missiles.  Flight 
testing  in  FY15  and  early  FY16  provided  additional  VV&A 
data,  but  the  BMDS  Operational  Test  Agency  has  not 
accredited  the  high  fidelity  modeling  and  simulation  suite 
for  performance  across  the  entire  design  battlespace. 

8.  Has  partially  addressed  the  fourth  recommendation  from 
FY14  to  conduct  sufficient  ground  and  flight  testing  of 
the  redesign  of  insulation  components  in  the  nozzle  of  the 
SM-3  Block  IB  TSRM  after  completion  and  installation 
of  the  new  design  concept  to  prove  the  new  design  works 
under  the  most  stressing  operational  flight  conditions,  when 
it  began  ground  testing  the  new  TSRM  nozzle  design. 

Flight  testing  is  planned  in  February  2016. 

FY15  Recommendations.  The  program  should: 

1.  Use  the  industry-led  FRB  process  to  identify  the  root 
cause  of  the  low  cold  gas  pressure  anomalies  from  recent 
lot  acceptance  testing  of  the  SM-3  Block  IB  CGR,  and 
determine  the  appropriate  corrective  actions  needed  to 
ensure  proper  functioning  of  that  SM-3  component. 
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2.  Conduct  stressing  simultaneous  air  and  ballistic  missile 
defense  engagements  with  the  Aegis  Afloat  system 
operating  in  Integrated  Air  and  Missile  Defense  radar 
priority  mode,  with  multiple  ballistic  missiles  and  anti-ship 
cruise  missile  threats  being  simultaneously  engaged. 


3.  Perform  high-fidelity  modeling  and  simulation  analysis 
over  the  expected  Aegis  Ashore  engagement  battlespace  for 
EPAA  Phase  2  to  allow  for  a  broad  quantitative  evaluation 
of  engagement  capability. 
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Command  and  Control,  Battle  Management,  and 
Communications  (C2BMC)  System 


Executive  Summary 

•  The  Missile  Defense  Agency  (MDA)  continued  to  mature 
the  Command  and  Control,  Battle  Management,  and 
Communications  (C2BMC)  system  with  the  implementation 
of  Spiral  6. 4-2. 2.0  (S6. 4-2. 2.0)  software  during  FY15.  The 
MDA  demonstrated  C2BMC  battle  management  functions 
during  ground  and  flight  tests  in  FY15  including  threat 
assessment,  threat  evaluation,  sensor  resource  management, 
sensor  track  data  processing,  track  reporting,  target  selection, 
sensor/weapon  access  determination,  and  engagement 
monitoring.  The  C2BMC  engagement  planner  provided 
non-real-time  performance  analysis  of  the  composition  and 
location  of  U.S.  and  allied  ballistic  missile  defense  (BMD) 
assets,  but  does  not  currently  provide  a  system-level  capability 
to  coordinate  engagement  decisions. 

•  C2BMC  exercised  additional  sensor  management,  track 
processing,  and  track  reporting  functionality  during  real-world 
targets  of  opportunity,  providing  valuable  insight  into  C2BMC 
operations  with  multiple  sensors,  networks,  coalition  partners, 
and  weapon  systems  outside  the  scope  of  flight  tests. 

•  C2BMC  continued  to  demonstrate  the  capability  for 
Combatant  Command  (CCMD)  sensor  managers  to  direct 
AN/TPY-2  Forward-Based  Mode  (FBM)  radars  to  execute 
focused  search  plans  and  respond  to  precision  cues  and 
reporting  of  system  track  data  during  ground  and  flight 
tests.  Ground  testing  demonstrated  automated  management 
of  multiple  AN/TPY-2  FBM  sensors.  Ground  testing 
further  demonstrated  the  successful  tasking/managing  of  the 
Kyoga-Misaki  AN/TPY-2  (FBM)  sensor  and  boost  phase 
cueing  between  multiple  AN/TPY-2  (FBM)  sensors. 

•  The  MDA  and  Red  Teams  from  the  Threat  Systems 
Management  Office  conducted  cyber  exercises  of  future 
spirals  of  the  Ballistic  Missile  Defense  System  (BMDS) 
elements  including  C2BMC  using  the  DOD  Enterprise 
Cyber  Range  Environment  (ECRE).  The  MDA  continues 
to  evaluate  the  results  of  this  event  to  identify  solutions 
to  reduce  the  likelihood  or  mitigate  the  effects  of  a  cyber 
intrusion.  The  MDA  teamed  with  U.S.  Cyber  Command’s 
(USCYBERCOM’s)  BMDS  national  Cyber  Protection  Team 
(CPT)  to  conduct  specialized  testing  at  C2BMC  suites  in 
U.S.  Central  Command  (USCENTCOM)  and  U.S.  European 
Command  (USEUCOM). 

System 

•  The  C2BMC  system  is  a  CCMD  interface  to  the  BMDS. 

More  than  70  C2BMC  workstations  are  fielded  at  U.S. 
Strategic  Command  (USSTRATCOM),  U.S.  Northern 
Command  (USNORTHCOM),  USEUCOM,  U.S.  Pacific 
Command  (USPACOM),  and  USCENTOM;  numerous 


Army  Air  and  Missile  Defense  Commands;  Air  and  Space 
Operations  Centers;  and  other  supporting  warfighter 
organizations. 

-  The  current  C2BMC  provides  CCMDs  and  other  senior 
national  leaders  with  situational  awareness  of  BMDS 
status,  system  coverage,  and  ballistic  missile  tracks  by 
displaying  selective  BMDS  data  for  strategic/national 
missile  defense  and  for  theater/regional  missile  defense, 
utilizing  multiple  message  formats  and  diverse  terrestrial 
and  satellite  communications  paths. 

-  The  C2BMC  also  provides  upper  echelon  planning  at  the 
CCMD  and  component  level.  BMDS  elements  (Aegis 
BMD,  Ground-based  Midcourse  Defense  [GMD],  Patriot, 
and  Terminal  High- Altitude  Area  Defense  [THAAD]) 
use  their  own  command  and  control  battle  management 
systems  and  mission-planning  tools  for  stand-alone 
engagements. 

-  The  C2BMC  S6.4  suite  provides  command  and  control  for 
the  AN/TPY-2  (FBM)  radar  as  well  as  track  reporting  to 
support  weapon  system  cueing  and  engagement  operations. 

•  Using  the  BMDS  Communications  Network,  the  C2BMC 
forwards  AN/TPY-2  (FBM)  and  AN/SPY-1  tracks  to  GMD. 
C2BMC  uses  the  Tactical  Digital  Information  Link-Joint 
message  formats  to  send  system  track  data  for  THAAD, 
Patriot,  and  coalition  system  cueing  and  for  Aegis  BMD 
engagement  support. 

•  The  C2BMC  S8.2  (projected  for  FY17/FY18)  is  intended 
to  mature  and  expand  S6.4  capabilities  as  the  next  major 
step  toward  integrated,  automated  sensor  management  and 
engagement  coordination. 
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Mission 

USSTRATCOM,  USNORTHCOM,  USEUCOM,  USPACOM, 
and  USCENTCOM  use  C2BMC  to  support  ballistic  missile 
defense  engagements.  CCMDs  use  C2BMC  for: 

•  Deliberate  and  dynamic  planning 

•  Situational  awareness 

•  Track  management 

•  AN/TPY-2  (FBM)  sensor  management  and  control 


•  Engagement  monitoring 

•  Data  exchange  between  C2BMC  and  BMDS  elements 

•  Network  management 

Major  Contractor 

Lockheed  Martin  Corporation,  Information  Systems  and  Global 
Solutions  -  Gaithersburg,  Maryland 


Activity 

•  The  MDA  conducted  C2BMC  system  and  sensors  testing 
during  FY15  in  accordance  with  the  DOT&E-approved 
Integrated  Master  Test  Plan. 

•  During  1QFY15,  the  MDA  and  Red  Teams  from  the  Threat 
Systems  Management  Office  conducted  cyber  exercises  of 
future  spirals  of  the  BMDS  elements,  including  C2BMC, 
using  the  DOD  ECRE.  The  MDA  continues  to  evaluate  the 
results  of  this  event  to  identify  solutions  and/or  procedures  to 
reduce  the  likelihood  of  occurrence  or  mitigate  the  effect  of 
a  cyber  intrusion.  The  MDA  teamed  with  USCYBERCOM’s 
BMDS  CPT  to  conduct  specialized  testing  at  C2BMC  suites  in 
USCENTCOM  and  USEUCOM. 

•  In  January  2015,  the  MDA  conducted  Ground  Test 
Distributed-04e  (GTD-04e)  Part  2,  an  operational  assessment 
of  new  USPACOM  and  USNORTHCOM  C2BMC  and 
AN/TPY-2  (FBM)  radar  functionality.  The  C2BMC 
managed  two  AN/TPY-2  (FBM)  radars  using  operational 
communications.  The  radars  detected  and  tracked  simulated 
intermediate-range  ballistic  missile  and  intercontinental 
ballistic  missile  threats  to  support  regional  and  strategic 
defense.  The  C2BMC  also  tasked  the  Kyoga-Misaki 
AN/TPY-2  (FBM)  radar,  managed  boost  phase  cueing  between 
the  two  AN/TPY-2  (FBM)  radars,  and  exercised  updates  to  the 
focused  search  plans  and  threat  enumerations. 

•  In  May  2015,  the  MDA  exercised  C2BMC  in  the  Ground  Test 
Integrated-06  (GTI-06)  Part  1  test,  a  hardware-in-the-loop 
event  to  provide  data  on  mutual  sensor  support  between 
multiple  AN/TPY  2  (FBM)  radars  equipped  with  updated 
search  plan  configurations  and  debris  mitigation  capabilities 
(radar  software  CX-2.1.0).  C2BMC  software  version 

S6. 4-2. 2.0  managed  multiple  AN/TPY-2  (FBM)  radar 
representations  in  a  cross-area  of  responsibility  (AOR)  data 
sharing  environment  between  USEUCOM  and  USCENTCOM. 
The  AN/TPY-2  (FBM)  radar  representations  detected  and 
tracked  simulated  medium-range  ballistic  missile  threats  and 
forwarded  the  track  data  to  the  C2BMC. 

•  The  BMDS  Operational  Test  Agency  and  the  MDA  attempted 
Flight  Test,  Operational-02  (FTO-02)  Event  1  in  June  2015 

at  the  Pacific  Missile  Range  Facility  on  Kauai,  Hawaii.  The 
MDA  intended  to  demonstrate  the  operational  capability  of  the 
regional/theater  European  Phased,  Adaptive  Approach  Phase  2 
BMDS,  anchored  by  the  Aegis  Ashore  Missile  Defense  System 
(AAMDS),  to  defend  Europe  against  medium-range  ballistic 


missiles.  The  test  was  to  be  the  first  target  intercept  by  the 
AAMDS  and  the  first  flight  for  the  Standard  Missile  (SM)-3 
Block  IB  with  threat  update  guided  missile.  C2BMC  software 
version  S6. 4-2. 2.0  was  to  have  managed  an  AN/TPY-2  (FBM) 
radar  (software  version  CX-2.1.0)  during  the  event.  Due  to  a 
target  malfunction,  the  test  was  not  completed  and  the  SM-3 
guided  missile  was  not  launched.  The  MDA  plans  to  attempt 
this  test  again  in  December  2015. 

•  In  July  2015,  C2BMC  software  version  S6. 4-2. 2.0  participated 
in  GTI-06  Part  3,  a  test  designed  to  assess  BMDS  performance 
and  interoperability  across  the  USNORTHCOM  mission 
space.  C2BMC  managed  one  AN/TPY-2  (FBM)  radar 
(software  version  CX-1.2.3)  and  demonstrated  track  data 
forwarding  to  GMD  Fire  Control  and  cueing  of  Aegis  BMD 
using  AN/TPY-2  (FBM)  track  data. 

•  The  BMDS  Operational  Test  Agency  and  the  MDA  conducted 
FTO-02  Event  2  and  Event  2a  in  September  and  October  2015 
at  Wake  Island  and  the  broad-ocean  area  surrounding 

it.  The  MDA  designed  the  test  mission  to  demonstrate 
a  layered  BMDS  with  multiple  combat  systems  sharing 
common  defended  areas  and  shot  opportunities  against 
two  threat-representative  ballistic  missiles.  The  primary 
test  objective  was  to  assess  Aegis  BMD  system  capability  to 
prosecute  a  ballistic  missile  threat  engagement  in  the  presence 
of  non-organic  post-intercept  debris,  while  simultaneously 
conducting  Anti-Air  Warfare.  The  THAAD  combat  system, 
using  Lot  4  interceptors  for  the  first  time,  generated  the 
non-organic  post-intercept  debris  scene  for  Aegis  BMD. 
C2BMC  software  version  S6. 4-2. 2.0  managed  one  AN/TPY-2 
(FBM)  radar,  track  reporting  of  sensor  data  to  Link  1 6,  and 
forwarded  track  data  between  Aegis  BMD  and  THAAD 
systems. 

•  At-Sea-Demonstration- 15  (ASD-15)  was  a  multinational 
interoperability  demonstration  of  coalition  Integrated  Air  and 
Missile  Defense.  The  event  used  new  NATO  data-sharing 
gateways  in  a  live  fire  event.  C2BMC  software  version 

S6. 4-2. 2.0  participated  as  a  data-sharing  component  between 
U.S.  Aegis  BMD  and  three  NATO  partners. 

•  The  BMDS  OTA  and  the  MDA  conducted  GTD-06  Part  1 
in  October  2015.  The  test  assessed  the  BMDS  European 
Phase,  Adaptive  Approach  Phase  2  architecture  against 
short-range  and  medium-range  ballistic  missile  engagements 
in  USEUCOM  and  USCENTCOM.  C2BMC  software  version 
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S6. 4-2. 2.0  managed  multiple  AN/TPY-2  (FBM)  radars  and 
exercised  cross-AOR  data  sharing. 

•  Throughout  FY15,  the  MDA  exercised  C2BMC  sensor 
management,  track  processing,  and  track  reporting  during 
real-world  targets  of  opportunity.  The  system  demonstrated 
dual  radar  management  and  track  processing/reporting 
utilizing  operational  C2BMC  suites  and  communications. 
These  events  also  demonstrated  USEUCOM-USCENTCOM 
cross-AOR  data  flows,  and  track  correlation  and  reporting 
for  AN/TPY-2  (FBM)  and  other  elements  (e.g.,  Aegis  BMD, 
Arrow,  and  Patriot). 

Assessment 

•  The  C2BMC  software  version  S6.4  Global  Engagement 
Manager  (GEM)  suite  provided  automated  management 
of  multiple  AN/TPY-2  (FBM)  radars  in  a  single  AOR  and 
enhanced  track  processing  and  reporting  while  requiring  less 
operator  involvement  as  compared  to  the  S6.2  software  and  the 
S6.4  Combatant  Command  (COCOM)  suite. 

•  C2BMC  demonstrated  dual  AN/TPY-2  (FBM)  sensor 
management  in  the  USPACOM  AOR  and  USEUCOM  AOR 
during  ground  tests  in  FY15.  Dual  radars  were  not  available 
for  assessment  during  flight  tests;  however,  C2BMC  did 
exercise  dual  radar  management,  precision  cueing,  and  system 
track  formation  during  real-world  targets  of  opportunity  in 
both  the  USPACOM  and  USEUCOM  AORs. 

•  During  GTD-04e  Part  2,  C2BMC  demonstrated  boost  phase 
cueing  capabilities  and  focused  search  plan  tasking  while 
managing  two  AN/TPY-2  (FBM)  radars  in  USPACOM,  and 
demonstrated  the  AN/TPY-2  (FBM)  radar’s  ability  to  support 
GMD  engagement  of  intermediate-range  ballistic  missile 
and  intercontinental  ballistic  missile  threats.  The  radars 
provided  data  on  the  simulated  missile  threats  to  the  C2BMC 
system  that  supported  BMDS  situational  awareness,  BMDS 
sensor  tasking,  and  GMD  engagement  planning.  C2BMC 
experienced  limitations  in  its  ability  to  task  focused  search 
plans  during  the  test.  The  MDA  is  addressing  the  focused 
search  plan  limitations  and  will  demonstrate  the  fixes  in 
GTI-06  Part  2  in  FY16. 

•  The  MDA  tested  new  data  exchange  conditions  between 
USEUCOM  and  USCENTCOM  during  GTI-06  Part  1  and 
assessed  data  on  new  cooperative  search  fences.  C2BMC 
generally  performed  nominally,  receiving  AN/TPY-2  (FBM) 
and  Link  16  data  and  forwarding  system  tracks  on  Link  16. 

In  addition,  real-world  events  demonstrated  NATO  and  Israel 


data  flows,  interoperability,  and  track  correlation  with  C2BMC 
that  could  not  be  assessed  in  the  ground  test. 

•  The  MDA  demonstrated  C2BMC  software  version  S6. 4-2. 2.0 
battle  management  functions  during  ground  and  flight  tests 
in  FY15  including  threat  assessment,  threat  evaluation, 
sensor  resource  management,  sensor  track  data  processing, 
track  reporting,  target  selection,  sensor/weapon  access 
determination,  and  engagement  monitoring.  The  C2BMC 
engagement  planner  provided  non-real-time  performance 
analysis  of  the  composition  and  location  of  U.S.  and  allied 
BMD  assets  but  does  not  currently  provide  a  system-level 
capability  to  coordinate  engagement  decisions.  CCMD 
sensor  managers  can  use  C2BMC  to  manually  task  a  single 
AN/TPY-2  (FBM)  radar  to  execute  focused  search  plans, 
respond  to  a  precision  cue,  or  automate  radar  tasking  of  up  to 
two  radars. 

•  During  test  execution  preparation  for  FTO-02  El,  a 
networking  problem  was  discovered  with  the  AN/TPY-2 
(FBM)  radar  software  version  CX  2.1.0  that  forced  the 
Soldiers  to  use  non-operational  radar  management  tactics, 
techniques,  and  procedures.  The  problem  was  fixed  and 
demonstrated  in  GTD-06  Part  1 . 

•  The  majority  of  FY15  testing  used  the  C2BMC  GEM 
suite.  In  the  event  of  a  GEM  suite  failure,  C2BMC  has  an 
option  to  use  the  COCOM  suite  for  management  of  a  single 
AN/TPY-2  (FBM).  This  option  was  tested  during  FTO-02 
Ela  hardware-in-the-loop  and  GTI-06  Part  1  testing  and 
demonstrated  nominal  performance. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  has 
addressed  12  of  13  previous  recommendations  for  the  C2BMC 
program  and  should  continue  to  include  cybersecurity 
assessments  in  BMDS-level  testing.  The  MDA  made  progress 
on  a  C2BMC  cyber  testing  strategy  by  performing  basic 
testing  and  system  scans  during  GTI-06  Part  1,  investigating 
and  demonstrating  cyber  testing  techniques  during  multiple 
ECRE  events,  and  teaming  with  USCYBERCOM’s  BMDS 
CPT  to  conduct  specialized  testing  at  C2BMC  suites  in 
USCENTCOM  and  USEUCOM. 

•  FY15  Recommendation. 

1 .  As  the  Warfighter  BMDS  concept  of  operations  continues 
to  evolve,  the  MDA  should  periodically  assess  the  COCOM 
suite  for  its  ability  to  provide  adequate  quality  of  service  in 
cases  of  GEM  suite  failure. 
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Ground-Based  Midcourse  Defense  (GMD) 


Executive  Summary 

•  Previous  assessments  that  the  Ground-based  Midcourse 
Defense  (GMD)  system  demonstrates  a  limited  capability 
to  defend  the  U.S.  Homeland  from  small  numbers  of 
intermediate-range  or  intercontinental  ballistic  missile  threats 
launched  from  North  Korea  or  Iran  remain  unchanged. 

•  The  Missile  Defense  Agency  (MDA)  did  not  conduct  GMD 
flight  testing  in  FY15.  The  GMD  system  did,  however, 
participate  in  two  BMDS  hardware-in-the-loop  ground  tests 
where  new  mission  functionality  of  the  GMD  Fire  Control 
(GFC)  software  and  interoperability  between  the  GMD  and 
Aegis  Ballistic  Missile  Defense  (BMD)  combat  systems  was 
demonstrated. 

•  The  Sea-Based  X-band  (SBX)  radar  acquired  and  tracked 
Minuteman  III  ballistic  missiles  during  three  Glory  Trip  (GT) 
missions. 

•  The  MDA  coordinated  and  received  approval  of  the 
Re-designed  Kill  Vehicle  Acquisition  Plan  and  emplaced 
eight  new  Capability  Enhancement-II  (CE-II)  Ground-Based 
Interceptors  (GBls)  in  the  Flight  Test,  GBI-06b  (FTG-06b) 
configuration. 

•  The  U.S.  Air  Force  became  the  lead  Service  for  the 
Long-Range  Discrimination  Radar  under  development  and 
selected  Clear  Air  Force  Station  (AFS),  Alaska,  as  its  future 
location. 

System 

GMD  is  a  BMDS  combat  system  that  counters 

intermediate-range  and  intercontinental  ballistic  missile  threats  to 

the  U.S.  Homeland.  The  GMD  consists  of: 

•  GBls  at  Fort  Greely,  Alaska,  and  Vandenberg  AFB,  California 

•  GMD  ground  system  including  GFC  nodes  at  Schriever 
AFB,  Colorado,  and  Fort  Greely,  Alaska;  Command  Launch 
Equipment  at  Vandenberg  AFB,  California,  and  Fort  Greely, 
Alaska;  and  In-Flight  Interceptor  Communication  System 
Data  Terminals  at  Vandenberg  AFB,  California;  Fort  Greely, 
Alaska;  and  Eareckson  Air  Station,  Alaska 

•  GMD  secure  data  and  voice  communications  system 
including  long-haul  communications  using  the  Defense 
Satellite  Communication  System,  commercial  satellite 
communications,  and  fiber-optic  cable  (both  terrestrial  and 
submarine) 

•  COBRA  DANE  Upgraded  Radar  at  Eareckson  Air  Station 
(Shemya  Island),  Alaska.  COBRA  DANE  is  a  fixed  site,  fixed 
orientation,  phased  array  L-band  radar  with  one  radar  face  that 
provides  120-degree  azimuth  field  of  view 

•  Upgraded  Early  Warning  Radars  (UEWRs)  at  Beale  AFB, 
California;  Royal  Air  Force  Fylingdales,  United  Kingdom; 
and  Thule  Air  Base,  Greenland.  These  sensors  are  fixed  site, 
fixed  orientation,  phased  array  ultra-high  frequency  radars. 
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The  radars  at  Beale  AFB  and  Thule  Air  Base  have  two  radar 
faces  that  provide  240-degree  azimuth  field  of  view;  the 
Flyingdales  radar  has  three  radar  faces  that  provide  a  full 
360-degree  field  of  view.  In  2012,  the  MDA  and  Air  Force 
Space  Command  awarded  a  contract  and  exercised  an  option 
to  upgrade  the  EWRs  at  Clear  AFS,  Alaska,  and  Cape  Cod 
AFS,  Massachusetts,  respectively. 

•  SBX  radar,  a  mobile  phased  array  sensor  operated  by  the 
MDA  and  located  aboard  a  twin-hulled,  semi-submersible, 
self-propelled,  ocean-going  platform 

•  External  interfaces  that  connect  to  Aegis  BMD;  North 
American  Aerospace  Defense/U.S.  Northern  Command 
Command  Center;  Command  and  Control,  Battle 
Management,  and  Communications  system  at  Peterson 
AFB,  Colorado;  Space-Based  Infrared  System/Defense 
Support  Program  at  Buckley  AFB,  Colorado;  and  AN/TPY 
2  (Forward-Based  Mode  (FBM))  radars  at  Japan  Air 
Self-Defense  Force  bases  in  Shariki  and  Kyoga-misaki,  Japan 

Mission 

Military  operators  from  the  U.S.  Army  Space  and  Missile 
Defense  Command/ Army  Forces  Strategic  Command  (the 
Army  service  component  to  U.S.  Strategic  Command)  will 
use  the  GMD  system  to  defend  the  U.S.  Homeland  against 
intermediate-range  and  intercontinental  ballistic  missile  attacks 
using  the  GBI  to  defeat  threat  missiles  during  the  midcourse 
segment  of  flight. 


GMD  371 


F  Y 1  5  BALLISTIC  MISSILE  DEFENSE  SYSTEMS 


Major  Contractors 

•  GMD  Prime:  The  Boeing  Company,  Network  and  Space 
Systems  -  Huntsville,  Alabama 

•  Boost  Vehicle:  Orbital  Sciences  Corporation,  Missile  Defense 
Systems  -  Chandler,  Arizona 

•  Exo-atmospheric  Kill  Vehicle:  Raytheon  Company,  Missile 
Systems  -  Tucson,  Arizona 


Fire  Control  and  Communications:  Northrop  Grumman 
Corporation,  Information  Systems  -  Huntsville,  Alabama 
COBRA  DANE,  UEWRs,  and  SBX:  Raytheon  Company, 
Integrated  Defense  Systems  -  Tewksbury,  Massachusetts 


Activity 

•  The  MDA  conducted  all  testing  in  accordance  with  the 
DOT&E-  approved  Integrated  Master  Test  Plan. 

•  The  MDA  did  not  conduct  GMD  interceptor  flight  testing 
in  FY15.  The  MDA  prepared  to  conduct  a  non-intercept 
Ground-based  Midcourse  Controlled  Test  Vehicle  Flight 
(GM  CTV-02+)  currently  scheduled  for  January  2016.  This 
test  will  verify  the  effectiveness  of  the  GBl’s  new  Alternate 
Divert  Thruster  and  collect  data  for  use  in  developing 
Mid-Term  Discrimination  Improvements  for  Homeland 
Defense. 

•  The  GMD  system  participated  in  two  BMDS 
hardware-in-the-loop  ground  tests. 

-  The  MDA  conducted  Ground  Test  Distributed-04e 
(GTD-04e)  Part  2  and  Ground  Test  lntegrated-06  (GT1-06) 
Part  3  in  January  and  July  2015,  respectively. 

-  The  MDA  used  hardware  and  software  representations 
of  the  GMD  system,  the  Space-Based  Infrared 
System/  Defense  Support  Program,  UEWRs,  C2BMC, 
AN/TPY-2  (FBM)  radar,  Aegis  BMD  AN/SPY- 1  radar  in 
its  long-range  surveillance  and  track  mode,  and  the  SBX 
radar  to  investigate  U.S.  Northern  Command  strategic 
scenarios  by  stimulating  the  BMDS  with  intelligence-based 
intercontinental  ballistic  missile  threats  launched  against 
the  U.S.  Homeland. 

-  The  COBRA  DANE  radar  completed  software  build 
2.7. 1.1  Operations  Trial  Period  Review  Panel  containing 
near-term  discrimination  improvements  in  March  2015  to 
support  GTI-06  Part  3  in  July  2015. 

•  The  SBX  Radar  exercised  detection  and  tracking  performance 
in  three  U.S.  Air  Force  intercontinental  ballistic  missile  GT 
missions:  GT  214  and  GT  215  in  March  2015,  and  GT  216  in 
October  2015. 

•  The  MDA  coordinated  and  received  approval  of  the 
Re-designed  Kill  Vehicle  Acquisition  Plan  and  emplaced 
eight  new  CE-II  GBls  in  the  FTG-06b  configuration. 

•  The  U.S.  Air  Force  became  the  lead  Service  for  the 
Long-Range  Discrimination  Radar  under  development  and 
selected  Clear  AFS,  Alaska,  as  its  future  location. 

Assessment 

•  Previous  assessments  that  state  the  GMD  demonstrates  a 
limited  capability  to  defend  the  U.S.  Homeland  from  small 


numbers  of  intermediate-range  or  intercontinental  ballistic 
missile  threats  launched  from  North  Korea  or  Iran  remain 
unchanged. 

•  In  GTD-04e  Part  2,  the  MDA  demonstrated  interoperability 
between  C2BMC  and  GFC  Build  6B2.2.  Further,  GFC  Build 
6B2.2  processed  data  from  the  new  AN/TPY-2  (FBM)  radar  in 
Kyoga-Misaki,  Japan. 

•  In  GTI-06  Part  3,  the  MDA  demonstrated  interoperability 
between  Aegis  BMD  4.0.3,  Aegis  BL9.C1,  Beale  UEWR 
9.0.4,  and  GFC  6B2.2  through  strategic  and  theater  scenarios 
and  supported  the  deployment  of  operational  assets  within 
the  BMDS  architecture  for  U.S.  Pacific  Command  and  U.S. 
Northern  Command. 

•  In  GT  214,  GT  215,  and  GT  216,  the  SBX  radar  acquired  and 
tracked  the  Minuteman  III  ballistic  missile  through  the  boost 
and  midcourse  phases  of  flight. 

•  The  selection  of  Clear  AFS,  Alaska,  for  the  future  location  of 
the  Long-Range  Discrimination  Radar  necessitates  examining 
if  additional  sensor  capability,  beyond  the  AN/TPY-2  (FBM) 
radars  located  in  Japan,  is  needed  to  defend  Hawaii. 

Recommendations 

•  Status  of  Previous  Recommendations.  The  MDA  has 
addressed  most  of  the  previous  FY14  recommendations. 
However,  the  following  remain  outstanding: 

1.  The  MDA  has  not  addressed  the  FY13  recommendation  to 
retest  the  Capability  Enhancement-I  exo-atmospheric  kill 
vehicle  in  order  to  accomplish  the  test  objectives  from  the 
failed  FTG-07  mission.  The  MDA  plans  to  address  this 
recommendation  in  4QFY17  during  FTG-11. 

2.  The  MDA  has  initiated,  but  not  completed,  the 
FY14  recommendation  to  extend  the  principles  and 
recommendations  contained  in  the  Independent  Expert 
Panel  assessment  report  on  the  GB1  fleet  to  all  Homeland 
Defense  components  of  the  BMDS. 

•  FY15  Recommendation. 

1 .  The  MDA  should  determine  any  additional  sensor 

capability  requirements  for  an  effective  Defense  of  Hawaii 
capability. 
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Terminal  High-Altitude  Area  Defense  (THAAD)  and 
AN/TPY-2  Radar  (Forward-Based  Mode) 


Executive  Summary 

•  The  Terminal  High- Altitude  Area  Defense  (THAAD)  Project 
Office  conducted  one  operational  flight  test  in  October  2015, 
in  accordance  with  the  DOT&E-approved  Integrated  Master 
Test  Plan,  intercepting  two  ballistic  missile  targets. 

•  The  Army  Test  and  Evaluation  Command  (ATEC)  conducted 
a  THAAD  Reliability  Growth  Test  (RGT)  January  2015 
through  March  2015,  which  collected  922  hours  of  reliability 
data  over  9  operational  periods.  This  testing  demonstrated 
some  increases  in  reliability,  with  the  notable  exception  of  the 
launcher  generators.  The  testing  also  revealed  problems  with 
the  radar-to-operator  interface. 

•  During  FY15,  THAAD  participated  in  several  Ballistic 
Missile  Defense  System  (BMDS)-level  ground  tests,  providing 
information  on  THAAD  functionality  and  interoperability  in 
various  theater  scenarios. 

•  The  THAAD  Program  Manager  continued  to  address  the 

18  material  release  conditions  that  need  to  be  resolved  before 
the  Army  will  grant  a  Full  Materiel  Release  for  the  first 
two  fire  units. 

System 

•  THAAD  is  intended  to  complement  the  lower-tier  Patriot 
system  and  the  upper-tier  Aegis  BMDS;  it  can  engage  threat 
ballistic  missiles  in  both  the  endo-  and  exo-atmosphere. 

•  THAAD  consists  of  five  major  components: 

-  Missiles 

-  Launchers 

-  AN/TPY-2  Radar  (Terminal  Mode) 

-  THAAD  Fire  Control  and  Communications 

-  THAAD  Peculiar  Support  Equipment 

•  THAAD  can  accept  target  cues  for  acquisition  from  Aegis 
BMD,  other  regional  sensors,  and  through  command  and 
control  systems. 

•  The  AN/TPY-2  (Terminal  Mode)  radar  is  used  by  THAAD. 
This  section  also  discusses  the  AN/TPY-2  Forward-Based 
Mode  (FBM)  variant  of  the  THAAD  radar.  Operated  by  the 
Army,  this  mobile/transportable  phased  array  X-band  radar 
provides  regional  and  strategic  ballistic  missile  threat  data  to 
the  entire  BMDS  through  the  Command  and  Control,  Battle 
Management,  and  Communications  (C2BMC)  system.  The 
AN/TPY-2  FBM  variant  of  the  THAAD  radar  is  currently 
deployed  in  Japan,  Israel,  Turkey,  and  the  U.S.  Central 
Command  area  of  responsibility. 


Mission 

•  U.S.  Strategic  Command  deploys  THAAD  to  protect 
critical  assets  worldwide.  U.S.  Northern  Command, 

U.S.  Pacific  Command,  U.S.  European  Command,  and 

U.S.  Central  Command  will  use  THAAD  to  intercept  short-  to 
intermediate-range  ballistic  missile  threats  in  their  areas  of 
responsibility. 

•  All  Combatant  Commanders  will  use  the  AN/TPY-2  (FBM) 
radar  to  detect,  track,  classify,  and  engage  ballistic  missile 
threats  that  target  the  United  States  and  its  allies,  and  to 
provide  data  for  situational  awareness  and  battle  management 
through  the  C2BMC. 

Major  Contractors 

•  Prime:  Lockheed  Martin  Corporation,  Missiles  and  Fire 
Control  -  Dallas,  Texas 

•  AN/TPY-2  Radar  (TM  and  FBM):  Raytheon  Company, 
Integrated  Defense  Systems  -  Tewksbury,  Massachusetts 


THAAD  373 


F  Y 1  5  BALLISTIC  MISSILE  DEFENSE  SYSTEMS 


Activity 

•  The  Missile  Defense  Agency  (MDA)  conducted  all  testing  in 
accordance  with  the  DOT&E-approved  Integrated  Master  Test 
Plan. 

•  ATEC  conducted  a  THAAD  RGT,  which  collected  922  hours 
of  reliability  data  over  9  operational  periods.  This  was  the 
first  test  conducted  with  the  new  Configuration  2  hardware 
and  software,  which  addresses  a  large  number  of  obsolescence 
issues.  ATEC  conducted  this  testing  January  2015  through 
March  2015  at  McGregor  Range,  New  Mexico. 

•  THAAD  provided  hardware-in-the-loop  representations  for 
two  BMDS-level  ground  tests:  Ground  Test  Distributed-04e 
(GTD-04e)  Part  2  in  January  2015,  using  version  2.2  software 
and  Ground  Test  lntegrated-06  (GTI-06)  Part  1  in  May  2015, 
using  version  2.2  software.  These  tests  provided  information 
on  THAAD  functionality  and  interoperability  in  various 
theater  scenarios. 

•  AN/TPY-2  (FBM)  participated  in  several  BMDS-level  ground 
tests.  Two  U.S.  Pacific  Command  AN/TPY-2  (FBM)  CX-1.2.3 
radars  participated  in  GTD-04e  Part  2  in  January  2015, 

HW1L  representations  of  the  AN/TPY-2  (FBM)  CX-2.1.0 
radars  were  used  in  the  GTI-06  Part  1  in  May  2015,  and 

a  HW1L  representation  of  an  AN/TPY-2  (FBM)  CX-1.2.3 
radar  was  used  in  the  GTI-06  Part  3  developmental  test.  In 
these  tests,  C2BMC  tasked  and  managed  the  Kyoga-Misaki, 
Japan  AN/TPY-2  (FBM)  radar,  two  AN/TPY-2  (FBM)  radars 
exercised  boost  phase  cueing,  and  C2BMC  managed  radar 
data  in  a  cross-area  of  responsibility  data  sharing  environment. 

•  The  THAAD  program  also  conducted  several  smaller  test 
events  including  missile  round  pallet  transportation  testing  in 
February  2015,  and  environmental  testing  of  the  radar  prime 
power  unit  in  June  2015. 

•  AN/TPY-2  (FBM)  CX-2.1.0  participated  in  Flight  Test 
Operational-02  (FTO-02)  Event  1  in  June  2015  at  the 
Pacific  Missile  Range  Facility  in  Kauai,  Hawaii.  The  MDA 
intended  to  demonstrate  the  operational  capability  of  the 
regional/ theater  European  Phased  Adaptive  Approach  Phase 
2  BMDS,  anchored  by  the  Aegis  Ashore  combat  system,  to 
defend  Europe  against  medium-range  ballistic  missiles.  The 
radar  would  have  cued  Aegis  Ashore  in  a  medium-range 
ballistic  missile  engagement.  Due  to  a  target  malfunction, 
MDA  did  not  complete  this  test  but  has  rescheduled  it  for  early 
FY16. 

•  The  MDA  conducted  BMDS-level  FTO-2  Event  2  and  Event 
2a  in  September  and  October  2015  at  Wake  Island  and  the 
broad-ocean  area  surrounding  it.  This  test  used  THAAD 
version  2.7  software  and  Lot  4  interceptors.  THAAD 
completed  near-simultaneous  engagements  of  two  targets: 

a  complex  short-range  ballistic  missile  target  and  a 
medium-range  ballistic  missile  target.  The  engagement  of  the 
medium-range  target  followed  the  failure  of  an  Aegis  BMD 
Standard  Missile-3  Block  IB  guided  missile  to  intercept  the 
target.  An  AN/TPY-2  (FBM)  radar  also  tracked  the  targets. 

•  The  MDA  continues  to  accumulate  contractor-collected 
reliability  data  and  score  the  data  for  the  deployed  AN/TPY-2 
(FBM)  radars.  The  THAAD  Program  Office  and  BMDS 


Operational  Test  Agency  are  also  preparing  for  a  90-day 
reliability  assessment  period  in  FY16  to  collect  and  compare 
reliability  across  the  fleet  of  deployed  AN/TPY-2  (FBM) 
radars. 

Assessment 

•  FTO-02  Event  2  and  Event  2a  demonstrated  that  THAAD, 
AN/TPY-2  (FBM),  and  BMDS  capabilities  against 
theater/regional  threats  increased  during  F Y 1 5  and  early 
FY16.  THAAD  Lot  4  interceptors,  for  the  first  time,  hit  one 
short-range  and  one  medium-range  threat-representative 
ballistic  missile  target  while  Aegis  BMD  simultaneously 
engaged  an  air-breathing  threat  with  Standard  Missile-2 
(SM-2)  Block  IIIA  guided  missiles.  An  SM-3  Block  IB  with 
Threat  Upgrade  guided  missile,  which  MDA  intended  to 

hit  the  medium-range  ballistic  missile  target,  failed  early  in 
flight.  Full  assessment  of  the  FTO-02  Event  2  and  Event  2a 
test  mission  data  with  respect  to  operational  effectiveness, 
operational  suitability,  and  interoperability  is  ongoing. 

•  ATEC  collected  RGT  reliability  data  in  a  full  battery 
configuration  in  an  operational  status  and  a  desert  environment 
with  the  new  THAAD  Configuration  2  hardware  and  software. 
In  general,  the  system  showed  signs  of  reliability  growth, 
although  there  were  a  few  notable  exceptions.  There  were 

an  inordinate  number  of  failures  of  the  launcher  generators. 
These  failures  do  not  necessarily  cause  system  aborts,  but  the 
maintenance  burden  if  these  problems  were  to  continue  would 
be  extremely  high.  ATEC  also  found  that  the  radar  does  not 
properly  alert  the  operators  of  the  operating  state  of  the  radar. 

•  During  GTD-04e  Part  2  and  GTI-06  Part  1,  the  BMDS 
Operational  Test  Agency  reported  several  findings  affecting 
both  THAAD  and  AN/TPY-2  (FBM)  that  require  further 
investigation.  These  findings  affect  volume  of  message  traffic, 
radar  resources,  operator  workload,  deficient  debris  mitigation 
implementation,  and  deficiencies  in  cybersecurity  scanning. 

•  The  THAAD  program  continued  work  on  achieving  a  Full 
Materiel  Release  of  the  first  two  THAAD  batteries,  which 
achieved  a  Conditional  Materiel  Release  in  February  2012. 

The  THAAD  Project  Office  continues  to  address  the  18  open 
conditions  that  need  to  be  resolved  before  the  Army  will 
grant  a  Full  Materiel  Release.  Fixes  and  testing  of  the  open 
conditions  are  scheduled  through  FY17.  Of  the  original  39 
conditions,  16  conditions  were  closed  in  FY12  -  FY14  and 

4  were  closed  in  FY15: 

-  Testing  of  the  optical  block  in  the  missile  Flight 
Sequencing  Assembly 

-  Establishing  an  institutional  training  base 

-  Conducting  a  system  supportability  demonstration  and 
modifying  procedures  based  on  the  demonstration,  as 
necessary 

-  Providing  and  validating  documented  procedures  for  the 
Contractor  Logistics  Support  personnel 

-  Work  also  continues  on  additional  materiel  release 
conditions  for  follow-on  THAAD  software  versions  1.3.1 
and  1.4.0. 
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•  The  Army  and  MDA  are  also  working  on  achieving  Full 
Materiel  Release  of  various  versions  of  the  AN/TPY-2  (FBM) 
software,  and  have  made  progress  this  year,  although  several 
of  the  conditions  will  take  years  to  close.  The  materiel  release 
conditions  for  version  CX1.2.3_18  also  includes  the  training 
and  documentation  required  for  Soldier  operation  of  the  radars. 
Soldiers  began  taking  over  some  operations  from  contractors 
on  some  of  the  radars  in  July. 

Recommendations 

•  Status  of  Previous  Recommendations.  DOT&E’s  classified 
February  2012  THAAD  and  AN/TPY-2  Radar  OT&E  and 
LFT&E  report  contained  7  recommendations  in  addition  to 
the  39  Conditional  Materiel  Release  conditions.  In  FY15,  the 
MDA  addressed  one  classified  recommendation  (Survivability 
#4)  of  the  five  remaining  recommendations.  The  MDA 
should  continue  to  address  the  two  remaining  classified 
recommendations  (Effectiveness  #2  and  Effectiveness  #5)  and 
the  two  remaining  unclassified  recommendations.  The  MDA 
and  the  Army  shoidd: 

1 .  Implement  equipment  redesigns  and  modifications 
identified  during  natural  environment  testing  to  prevent 
problems  seen  in  testing  (Suitability  #1 1).  Some,  but  not 
all,  of  these  deficiencies  have  been  addressed  by  hardware 
modifications  included  in  THAAD  Configuration  2. 
Conducting  additional  ground  testing  with  Configuration  2 


(a  standing  FY14  recommendation)  would  also  provide  data 
to  address  this  recommendation. 

2.  Conduct  electronic  warfare  testing  and  analysis 

(Survivability  #3).  The  MDA  conducted  preliminary  testing 
during  FY13,  but  additional  testing  is  required. 

-  The  program  addressed  the  FY13  recommendation  to 
reassess  their  reliability  and  maintainability  growth 
planning  curves. 

-  The  program  partially  addressed  the  FY14 
recommendation  to  conduct  thorough  end-to-end  testing  of 
the  THAAD  Configuration  2  that  incorporates  considerable 
obsolescence  redesigns  of  hardware  and  software.  The 
MDA  should  continue  to  plan  to  rigorously  ground-test  the 
THAAD  system  to  verify  that  these  changes  can  withstand 
the  range  of  environments  and  conditions  required. 

•  FY15  Recommendation. 

1 .  The  MDA  should  prioritize  flight  and  ground  testing  that 
involves  THAAD  and  Patriot  engagement  coordination  to 
ensure  that  information  passed  between  THAAD  and  Patriot 
does  not  disrupt  organic  intercept  capabilities  or  contribute 
to  increased  interceptor  wastage  and  threat  missile  leakage. 
The  Integrated  Master  Test  Plan  17.1  includes  Patriot  and 
THAAD  participation  during  the  FTO-03  operational  flight 
test. 
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Live  Fire  Test  and  Evaluation  (LFT&E) 


Introduction 

•  In  F Y 1 5,  DOT &E  executed  LFT &E  oversight  for 

121  acquisition  programs,  3  LFT&E  investment  programs 
(Joint  Technical  Coordinating  Group  for  Munitions 
Effectiveness  (JTCG/ME),  Joint  Aircraft  Survivability 
Program  (JASP),  and  Joint  Live  Fire  (JLF)),  and  3  special 
interest  programs  (Warrior  Injury  Assessment  Manikin 
(WIAMan),  Home  Made  Explosives  (HME),  and  Small  Boat 
Shooters’  Working  Group). 

•  In  support  of  a  range  of  acquisition  decisions  and  activities, 
DOT&E  published  seven  LFT&E  reports  and  six  combined 
OT&E  and  LFT&E  reports.  The  reports  include 
recommendations  to  the  Services  to  further  improve  the 
survivability  and  lethality  of  the  subject  systems  for  a  range 
of  operationally  relevant  scenarios  in  existing  and  expected 
combat  environments. 

•  JTCG/ME  continued  to  develop  and  standardize 
methodologies  for  evaluating  munitions  effectiveness, 
including  target  vulnerability  characterization,  munitions 
lethality,  weapon  system  accuracy,  and  specific  weapon-target 
pairings  driven  primarily  by  operational  lessons  learned 
(Enduring  Freedom,  Iraqi  Freedom,  Odyssey  Dawn,  and 
Inherent  Resolve),  Joint  Staff  Data  Call,  and  the  needs 

of  Combatant  Commands.  The  two  major  JTCG/ME 
weaponeering  products  are  the  Joint  Munitions  Effectiveness 
Manual  (JMEM)  Weaponeering  System  (JW  S)  and 
Joint- Anti-air  Combat  Effectiveness  (J-ACE).  JTCG/ME  is 
currently  developing  a  third  weaponeering  product,  a  JMEM 
for  cyberspace  operations.  The  JWS  and  J-ACE  enabled: 

-  Ongoing  Combatant  Commands’  operational  targeting, 
weaponeering,  and  collateral  damage  estimation  calls 
in  direct  support  of  operations,  mission  planning,  and 
training;  warfighters  were  able  to  put  ordnance  on  target 
and  as  such,  directly  affect  combat  effectiveness  and  the 
war  against  terrorism. 

-  The  Air  Warfare  community,  in  particular  the  Navy  Strike 
Fighter  Weapons  School  and  the  Air  Force  Weapons 
School,  to  develop  tactics,  techniques,  and  procedures 
(TTP)  manuals  for  air  superiority  applications  and  to 
perform  post-shot  analysis  of  missile  firings  following  a 
training  mission. 

-  The  onset  of  the  development  of  building  blocks  for  a 
Cyber  JMEM  (CJMEM). 

-  DOD,  joint,  and  Service  planners  for  force-on-force 
modeling,  mission  area  analysis,  requirements  studies,  and 
weapon  procurement  planning. 

-  The  acquisition  community  in  performance  assessment, 
analysis  of  alternatives,  and  survivability  enhancement 
studies. 

-  United  Kingdom,  Canada,  Australia,  and  other 
coalition  partners  to  plan  operational  weaponeering  and 


collateral  damage  estimates,  support  training  and  tactics 
development,  and  support  force-level  analyses. 

•  JASP  funded  55  multi-year  projects  addressing  aircraft 
survivability  technologies  and  aircraft  survivability 
evaluation  tools.  JASP’s  primary  mission  is  to  increase 
combat  effectiveness  of  U.S.  military  aircraft  in  current  and 
emerging  threat  environments  through  joint  and  Service  staff 
coordination  and  development  of  survivability  technologies 
and  assessment  methodologies.  In  FY15,  JASP  made  progress 
in  improving: 

-  Aircraft  ability  to  counter  near-peer  and  second-tier 
threats  by  assessing  innovative  electro-optical  and 
infrared  countermeasures  (EO/IRCM)  and  radio  frequency 
countermeasures  (RFCM). 

-  Aircraft  force  protection  by  (1)  addressing  the  ability  to 
avoid  threat  detection/engagement,  e.g.,  Hostile  Fire  (HF) 
detection,  identification,  and  geolocation  technologies 

to  improve  aircrew  situational  awareness,  and  (2)  by 
implementing  aircraft  hardening  technologies,  e.g., 
armor  solutions,  self-sealing  fuel  tanks,  and  improved 
crashworthiness  technologies  including  improved 
helicopter  seats. 

-  Aircraft  survivability  to  fires,  the  primary  threat-induced 
aircraft  vulnerability. 

-  The  capabilities  of  survivability-related  models  with  the 
inclusion  of  emerging  threats,  by  automating  the  analysis 
and  post-processing,  and  by  continuing  to  validate  the  new 
and  existing  model  capabilities. 

•  JLF  supplemented  LFT&E  of  fielded  systems,  addressed 
operational  commander’s  needs,  and  characterized  new 
survivability  and  lethality  effects  of  fielded  systems  in 
response  to  exposure  of  U.S.  systems  to  new  threats  or  as  a 
result  of  systems  being  used  in  new,  unanticipated  ways,  or 
operated  in  new  environments.  Specifically,  JLF : 

-  Assessed  the  impact  of  fielded  system  design  changes  on 
survivability  (e.g.,  rotary-wing  aircraft  with  added  internal 
auxiliary  fuel  tanks) 

-  Assessed  weapon  lethality  against  new  targets  (e.g.,  fast 
attack  craft,  a  new  threat  to  U.S.  ships) 

-  Improved  accuracy  and  fidelity  of  weapon  data  used  as  part 
of  mission  planning  to  estimate  effectiveness  with  higher 
confidence  (e.g.,  improved  collateral  damage  estimates) 

-  Advanced  live  fire  test  methodologies  to  keep  pace  with 
changing  threats 

-  Supported  the  development  and  improvement  of  modeling 
and  simulation  tools  that  contribute  to  survivability 

and  lethality  evaluations  (e.g.,  new  data  to  support 
improvements  in  predicting  weapons  effects  against 
aircraft,  vehicles,  and  military  structures) 
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LFT&E  continued  its  oversight  of  three  special  interest 

programs: 

-  The  Warrior  Injury  Assessment  Manikin  (WIAMan) 
project,  an  Army-led  effort,  made  significant  progress  in 
biomechanics  testing  and  anthropomorphic  test  device 
(ATD)  development  to  design  a  biofidelic  prototype 

for  assessing  injuries  to  vehicle  occupants  during  the 
underbody  blast  (UBB),  but  the  Army  has  not  programmed 
any  funding  for  this  project  in  FY17  or  beyond. 

-  The  Home  Made  Explosives  Characterization  program 
(HME-C)  completed  multiple  test  phases,  intended  to 
investigate  the  repeatability  of  HME  surrogate  effects 
relative  to  those  of  TNT  and  the  effects  of  soil  condition 
and  IED  emplacement  on  HME  threat  performance. 

-  The  Small  Boat  Shooters’  Working  Group  continues  to 
synchronize  live  fire  and  other  operational  test  approaches 

against  this  growing  threat  class,  which  operates  in  littoral 
waters. 

LFT&E  OVERSIGHT 

The  primary  objective  of  LFT&E  is  to  evaluate  the  survivability 
and  lethality  of  acquisition  programs  and  to  identify  deficiencies 
to  be  corrected  before  those  platforms  or  munitions  enter  full-rat 
production.  In  FY15,  DOT&E  executed  LFT&E  oversight 
for  122  acquisition  programs.  Of  those,  21  operated  under 
the  waiver  provision  of  U.S.  Code,  Title  10,  Section  2366,  by 
executing  an  approved  alternative  LFT&E  strategy  in  lieu  of 
full-up  system-level  testing.  DOT&E  published  seven  LFT&E 
reports  and  six  combined  OT&E  and  LFT&E  reports  on  the 
following  programs  during  the  past  year:1 

1  Reports  marked  with  an  asterisk  were  sent  to  Congress. 

LFT&E  Reports 

Combined  OT&E  and  LFT&E  Reports 

Interim  Report  on  the  LFT&E  of  the  Hellfire  Missile  Variant 

Aegis  Ballistic  Missile  Defense  (BMD)  4.0  and  Standard  Missile-3 
Block  1 B* 

Stryker  Reactive  Armor  Tiles  (SRAT)  II  Live  Fire  Test  &  Evaluation* 

Lot  4  AH-64E  Apache  Attack  Helicopter  with  classified  annex* 

Cartridge  7.62  Ball  M80A1  Live  Fire  Test  &  Evaluation  Report* 

Guided  Multiple  Launch  Rocket  System  -  Alternative  Warhead 
(GMLRS-AW)* 

MaxxPro  Dash  with  Independent  Suspension  System  (ISS)  and  Maxx 
Pro  Survivability  Upgrade 

GBU/53B  Small  Diameter  Bomb,  Increment  II 

Hellfire  Final  Lethality  Report* 

Mobile  Landing  Platform  with  Core  Capability  Set  (MLP  with  CCS)  and 
classified  annex* 

Littoral  Combat  Ship  (LCS)  3Total  Ship  Survivability  Trial  (TSST) 

Air  Intercept  Missile  -  9X  (AIM-9X)  Block  II  (with  appendices)* 

Joint  Light  Tactical  Vehicle  Live  Fire  Test  &  Evaluation  Report 

Four  reports  supported  Full-Rate  Production  decisions: 

-  Aegis  Ballistic  Missile  Defense  (BMD)  4.0  and 

Standard  Missile-3  Block  IB  reported  on  the  lethality 
of  the  Standard  Missile-3  Block  IB  and  included 

two  recommendations  to  improve  future  evaluations  of 

BMD  lethality. 

-  Guided  Multiple  Launch  Rocket  System  -  Alternative 
Warhead  (GMLRS-AW)  reported  critical  weapon  lethality 
data  as  a  function  of  target  types,  target  areas,  target 
location  error,  and  countermeasures.  LFT&E  made 
three  recommendations  to  improve  weapon  lethality 
including  new  targeting  procedures  and  reassessment  of 
the  weapon  effectiveness  requirements  to  ensure  they 
adequately  represented  warfighters’  mission  success  needs. 

-  Mobile  Landing  Platform  with  Core  Capability  Set  (MLP 
w/CCS)  confirmed  the  survivability  shortfalls  of  this  ship 
(built  to  commercial  standards)  including  the  lack  of  hull 
and  equipment  hardening  or  personnel  protection  features 
necessary  to  survive  enemy  weapon  effects.  LFT&E 
identified  the  limitation  in  data  needed  to  assess  the 
effectiveness  of  the  Embarked  Security  Teams  for  close-in 
self-defense. 

-  Air  Intercept  Missile-9X  (AIM-9X)  Block  II  reported 
on  the  effectiveness  and  lethality  of  the  Block  II  missile. 

LFT&E  assessed  that  new  improvements  to  the  AIM-9X 
Block  II  fuze  did  not  degrade  the  missile’s  lethality  or 
effectiveness  compared  to  existing  AIM-9X  Block  I 
missiles. 

•  Five  reports  supported  a  program  decision: 

-  Cartridge  7.62  Ball  M80A1  LFT&E  assessed  the  lethality 
capability  of  the  new  7.62  mm  cartridge,  including 
effective  range,  as  fired  from  two  different  weapons  and 
against  a  range  of  targets  of  interest  (e.g.,  soft  targets, 
representative  battlefield  barriers,  and  personnel  protectioi 
equipment).  LFT&E  identified  additional  operationally 
relevant  targets  that  should  be  assessed  for  similar 
munitions  in  future  programs. 

-  Joint  Light  Tactical  Vehicle  (JLTV)  LFT&E  provided 
critical  survivability  information  to  the  procurement 
decision  makers,  with  a  focus  on  the  ability  of  each  of 
the  three  vendor  JLTV  prototypes  to  provide  protected 
ground  mobility  for  Soldiers  and  Marines  in  a  combat 
environment.  LFT&E  enabled  a  performance  comparison 
among  the  three  vendors  and  against  legacy  vehicles,  and 
identified  recommendations  specific  for  each  vendor  to 
help  improve  crew  and  vehicle  survivability. 

-  GBU/53B  Small  Diameter  Bomb,  Increment  II  (SDB  11) 
provided  critical  program  status  information  to  the 
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acquisition  officials  on  SDB  H’s  lethality  and  effectiveness 
when  employed  in  the  normal  attack  mode.  LFT&E 
provided  probability  of  single  shot  kill  information 
for  a  range  of  moving  and  stationary,  operationally 
representative  targets,  as  a  function  of  weapon  accuracy 
and  end-game  geometry. 

-  Lot  4  AH-64E  Apache  Attack  Helicopter  FOT &E  with 
classified  annex  provided  a  survivability  assessment 
of  the  Lot  4  AH-64E.  LFT&E  evaluated  AH-64E 
survivability  against  the  range  of  tested  threats  and 
recommended  improvements  and  upgrades  to  several  key 
survivability-related  systems. 

-  The  interim  report  on  the  LFT&E  of  the  Hellfire  Romeo 
Missile  Variant  provided  the  Program  Office  with  critical 
weapon  lethality  assessment  when  fired  from  unmanned 
aerial  vehicles  against  a  range  of  enemy  targets  of  interest. 
LFT&E  made  four  recommendations  to  further  improve 
the  understanding  of  the  missile’s  capability  against  very 
specific  targets  of  interest  or  as  fired  from  additional 
platforms. 

•  Four  reports  provided  system  survivability  or  lethality 

evaluations  for  use  by  the  Service  and  Program  Office: 

-  Stryker  Reactive  Armor  Tiles  (SRAT)  II  LFT&E  provided 
critical  data  on  the  survivability  of  Stryker  vehicles 
equipped  with  SRAT  II.  LFT&E  provided  multiple 
recommendations  to  the  Army  to  improve  survivability  of 
SRAT  II-equipped  vehicles,  as  well  as  lessons  learned  for 
future  LFT&E  efforts  involving  complex  armors. 


-  MaxxPro  Dash  with  Independent  Suspension  System 
(ISS)  and  Maxx  Pro  Survivability  Upgrade  evaluated  the 
response  of  the  vehicle  and  subsequent  protection  of  its 
occupants,  Soldiers  and  Marines,  to  attacks  as  those  seen  in 
Operation  Iraqi  Freedom  and  Operation  Enduring  Freedom. 
LFT&E  demonstrated  that  the  survivability  upgrades 
provided  significant  improvement  in  force  protection  and 
set  the  standards  for  Mine  Resistant  Ambush  Protected 
(MRAP)-level  underbody  blast  protection.  LFT&E  also 
provided  four  recommendations  to  further  improve  crew 
protection  and  fuel  fire  mitigation. 

-  Littoral  Combat  Ship  (LCS)  3  Total  Ship  Survivability 
Trial  (TSST)  confirmed  significant  vulnerabilities  in  the 
Freedom-class.  LFT&E  provided  insight  into  design 
changes  to  reduce  ship  vulnerability  and  improve 
recoverability.  LFT&E  identified  several  components  and 
systems  that  could  be  redesigned  or  reconfigured  to  make 
the  ship  more  survivable  without  requiring  major  structural 
modifications. 

-  The  final  lethality  report  on  the  Hellfire  missile  provided 
additional  evaluation  of  weapon  lethality  against  specified 
maritime  targets. 

•  DOT&E  published  one  classified  Special  Report,  the  report 
on  the  LCS  required  by  Section  123  of  H.R.  3979,  National 
Defense  Authorization  Act  for  FY15. 


LFT&E  INVESTMENT  PROGRAMS 


JOINT  TECHNICAL  COORDINATING  GROUP  FOR  MUNITIONS 
EFFECTIVENESS  (JTCG/ME) 

The  Joint  Technical  Coordinating  Group  for  Munitions 
Effectiveness  (JTCG/ME)  continued  to  update  and  develop 
weapons  and  targets  data  and  methodologies  crucial  for 
the  development  of  force  employment  options  for  theater 
commanders  and  the  resulting  execution  tasking  orders  for 
tactical  units.  The  principal  products  of  the  JTCG/ME  are 
the  Joint  Munitions  Effectiveness  Manuals,  or  JMEM.  The 
JMEM  include  detailed  data  on  the  physical  characteristics  and 
performance  of  weapons  and  weapon  systems;  descriptions  of  the 
mathematical  methodologies  that  employ  these  data  to  generate 
effectiveness  estimates;  software  that  permit  users  to  calculate 
effectiveness  estimates;  and  pre-calculated  weapon  effectiveness 
estimates.  It  permits  a  standardized  comparison  of  weapon 
effectiveness  across  all  Service  communities.  All  JMEM  weapon 
effectiveness  products  are  integrated  into  a  single  program,  the 
JMEM  Weaponeering  System  (JWS),  which  includes  the  Joint 
Anti-Air  Combat  Effectiveness  (J-ACE)  product.  The  JWS  is 
target  oriented  allowing  users  to  adequately  plan  the  mission 
by  determining  the  effectiveness  of  weapon  systems  against  a 
specified  target  for  a  range  of  weapon  delivery  modes. 


Joint  Munitions  Effectiveness  Manual  (JMEM)  Weaponeering 
System  (JWS) 

JWS  is  the  DOD  source  for  air-to-surface  and  surface-to-surface 
weaponeering,  munitions,  and  target  information  used  daily  in 
the  U.S.  Central  Command  (USCENTCOM)  and  U.S.  Africa 
Command  (USAFRICOM)  Areas  of  Responsibilities  (AORs) 
in  the  deliberate  planning  process  directly  supporting  Joint 
Publication  3-60  “Joint  Targeting.” 

JWS  enables  Combatant  Commands  to  efficiently  prosecute  their 
target  sets.  JWS  incorporates  accredited  methodologies,  certified 
munition  characteristics,  delivery  accuracy,  target  vulnerability 
data,  and  numerous  user  aids  to  support  the  operational  use  of 
JWS  to  predict  weapons  effectiveness  for  fielded  weapons  and 
delivery  systems.  JWS  is  the  calculation  engine  used  to  develop 
Quick  Weaponeering  Guides/Probability  of  Kill  Lookup  Tables  to 
address  time  sensitive  targets. 

In  FY15,  in  support  of  operational  commanders,  targeteers, 
weaponeers,  and  planners,  the  JTCG/ME  released  JWS  v2.2  that 
included  Digital  Precision  Strike  Suite  (DPSS)  Collateral  Damage 
Estimation  (DCiDE)  Tool  Version  1.2.2  and  Collateral  Damage 
Estimation  (CDE)  Tables.  JWS  v2.2  includes  approximately  220 
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methodology,  functionality,  weapon/ warhead/ fiize,  and  target 
updates.  Development  of  JWS  v2.2  is  now  complete.  As  a 
result.  Combatant  Commands  have  access  to: 

-  Additional  weapon  data  updates  such  as  GBU-49; 
Advanced  Precision  Kill  Weapon 
System-II;  HELLFIRE  variants;  Joint 
Air-to-Surface  Standoff  Missile  (JASSM) 

Delivery  Accuracy;  and  M982  Excalibur 
traj  ectory/ accuracy. 

-  Approximately  50  new  or  updated  materiel 
targets,  e.g.,  new  building  types  such  as 
brick  office,  pre-cast  walk slab  office, 
earth-timber  command  post,  etc.,  and  new 
quasi-static  blast  capability. 

-  Increased  mission  planning  efficiency 
through  the  inclusion  of  an  initial  DCiDE 
connectivity  that  improves  both  speed  and 
throughput  of  data,  as  shown  in  Figure  1. 

•  The  JTCG/ME  released  DCiDE  vl.2.2  with 
enhancements  to  directly  support  the  Chairman 
of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 

3160. 01A-  “No-Strike  and  the  Collateral  Damage 
Estimation  (CDE)  Methodology.” 


guidance  and  direction  from  the  Joint  Staff,  the  JWS  v2.2 
and  future  versions  will  be  released  to  several  key  coalition 
partners  in  support  of  current  operations  under  Foreign 
Military  Sales  agreements.  This  capability  is  critical  to  the 


Supports  Multiple 
Targets  in  Scenario 


Import  the  Target  into  JWS 


Bound  the  Target  in  DCiDE 


Modify  Building  Properties 


Transfer  Solution  to  DCiDE 


Edit  Weapon  Aimpoints  View  Results 

Figure  1.  Connectivity  between  Weaponeering  and  Collateral  Damage  Assessment 
Enables  Combatant  Commanders  to  More  Rapidly  Prosecute  Targets 


-  The  DCiDE  tool  is  critical  to  the  warfighters’  ability 
to  meet  urgent  operational  needs  for  an  accredited 
automated  CDE  tool  that  both  expedites  and  simplifies  the 
CDE  process.  DCiDE  is  the  only  automated  CDE  tool 
authorized  for  use  in  the  USCENTCOM  and  USAFRICOM 
AORs.  The  JTCG/ME  CDE  tables  are  used  in  every 
planned  kinetic  strike  in  all  AOR’s  to  meet  Commanders 
intent  and  to  minimize  civilian  casualties.  DOT&E  has 
received  positive  feedback  on  the  use  of  the  Collateral 
Effect  Radii  (CER)  values  as  a  critical  enabler  in  support  of 
munitions  employment  against  High  Value  Targets  (HVTs). 

-  JTCG/ME  accredited  CER  Reference  Tables  for 
air-to-surface  and  surface-to-surface  weapons,  which 
are  the  basic  data  that  supports  the  CDE  methodology. 
Changes  included  additions  for  air  burst  munitions 
and  nomenclature  changes.  Additional  updates  have 
been  provided  for  newly  fielded/updated  systems, 

e.g.,  GBU-49/BLU-133;  AGM-176A;  155  mmM109A, 
M549A1,  and  M795  with  Guided  M1156  Precision 
Guidance  Kit  (PGK)  Fuze.  In  support  of  advanced  CDE 
mitigation  techniques,  JTCG/ME  also  developed  the 
Collateral  Effects  Library  Tool. 

-  The  JTCG/ME  trained  multiple  users  at  different 
Commands  to  support  CDE  decisions.  Specifically  the 
JTCG/ME  trained  warfighters  from  the  III  Corps  (G2  and 
FIRES),  82nd  Airborne  Division,  3rd  Special  Forces  Group 
(SFG),  5th  SFG,  Task  Force  (TF)  3-10,  and  CENTCOM 
Joint  Targeting  Element  (JTE)  in  support  of  Operation 
Inherent  Resolve. 

JTCG/ME  reviewed  and  remarked  JWS  v2.2  to  ensure  all  data 
were  disclosable  to  facilitate  coalition  interoperability.  Risk 
Management  Framework  testing  is  underway  to  facilitate 
release  of  the  product  to  the  field.  Based  on  the  current 


effectiveness  of  U.S.  targeting  and  fires  personnel  working  in 
the  combined  environment. 

•  JWS  v2.3  is  under  development  and  will  include  an  interface 
to  implement  improved  aimpoint  development,  which  includes 
fields  for  weaponeering,  CDE,  and  precision  point  mensuration 
(PPM).  This  data  standard  is  currently  in  fielded  mission 
planning  systems.  JWS  v2.3  will  also  add  an  updated  Gunship 
Delivery  Accuracy  Program  module,  Rotary-Wing  Delivery 
Accuracy  Program,  and  Fast  Integrated  Structural  Tool  ground 
shock  kill  updates  that  will  improve  effectiveness  estimates. 

•  JTCG/ME  is  initiating  efforts  to  support  target  production 
activities  by  enabling  the  automated  integration  of 
weaponeering,  PPM,  and  CDE.  These  efforts  will  improve 
both  timeliness  and  data  throughput  associated  with  these 
activities.  In  addition,  connectivity  to  mission  planning 
systems  (Joint  Targeting  Toolbox)  and  databases  (Modernized 
Integrated  Database)  is  underway  to  allow  data  to  flow 
seamlessly  within  the  joint  targeting  processes. 

Joint- Anti-air  Combat  Effectiveness  (J-ACE) 

J-ACE  is  used  by  U.S.  Strategic  Command  (USSTRATCOM)  in 
the  support  of  route  planning  for  the  execution  of  strike  packages. 
J-ACE  simulates  air-to-air  and  surface-to-air  engagements.  It 
includes  accredited  blue/red/gray  (friendly/adversary/neutral) 
air-to-air  missile  (AAM)  models  and  red/gray  surface-to-air 
missile  (SAM)  fly-out  models  to  provide  probability  of  kill 
estimates.  J-ACE  is  the  umbrella  program  that  includes  both 
the  Joint  Anti-air  Model  (JAAM)  and  Endgame  Manager,  which 
provides  a  full  kill  chain  end-to-end  capability  for  mission 
analysis,  tactics  development,  and  training. 

•  J-ACE  v5.3,  currently  under  development,  will  provide 
extended  and  updated  data  sets  for  blue  missile  and  aircraft 
target  aero-performance,  anti-air  missile  lethality,  and  red  air 
target  vulnerability.  In  particular,  new  or  updated  air-to-air  or 
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surface-to-air  government-furnished  missile  or  weapon  fly  out 
models  will  be  integrated. 

•  The  JAAM  (Figure  2)  is  integrated  into  automated  systems 
used  directly  by  the  Air  Warfare  community,  in  particular 
the  Navy  Strike  Fighter  Weapons  School  and  the  Air  Force 
Weapons  School,  to  develop  TTP  manuals  for  air  superiority 
applications  and  to  perform  post-shot  analysis  of  missile 
firings 
following 
training 
missions. 

The 
JAAM 
is  being 
updated 
to  include 

the  Figure  2.  The  primary  J-ACE  interface  is  through  the 

effect  of  Joint  Anti-Air  Model  (JAAM).  JAAM  is  a  fast  running 

simulation  of  Air-to-Air  Missiles  (AAM)  and  Surface-to-Air 
weapon  Missiles  (SAM);  and,  aircraft  aerodynamic  performance. 

system 

reliability  on  the  probability  of  a  successful  engagement  (i.e.,  a 
reliability  factor  is  applied  to  the  weapon  system,  prior  to 
finishing  the  tactics  development;  it  is  usefid  to  understand 
outcomes  and  tactical  options  when  shots  fail  to  achieve  the 
desired  effectiveness). 

Joint  N on-Kinetic  Effects  -  Cyber  JMEM 
JTCG/ME  is  continuing  the  development  of  non-kinetic  tools  and 
capabilities,  the  Joint  Non-Kinetic  Effects.  Joint  Non-Kinetic 
Effects  is  intended  to  be  a  single  source  for  operational 
warfighters,  analysts,  targeteers,  and  planners  to  analyze 
offensive  cyber  capability  effectiveness. 

•  In  conjunction  with  the  Air  Force  Targeting  Center,  the 
JTCG/ME  is  developing  preliminary  JMEMs  for  cyberspace 
operations.  Current  efforts  are  focused  on  developing  the 
building  blocks  for  a  Cyber  JMEM  (CJMEM)  including 
weapons  characteristics,  target  vulnerability,  and  effects 
estimation  tools  (e.g.,  Cyber  Capabilities  Registry  (CCR), 
Cyber  Critical  Elements/Weaponeering  Guide).  In 

FY15,  JTCG/ME  made  progress  in  developing  weapons 
characterization  data  and  testing  standards. 

Operational  Users  Working  Group 

The  Operational  Users  Working  Group  (OUWG)  is  a  critical 
venue  for  receiving  direct  user  feedback  and  development  of 
future  requirements  from  the  operational  community  in  regards 
to  needed  software  enhancements  and  capabilities  to  support 
air-to-surface  and  surface-to-surface  target  engagements. 
Examples  of  user  requirements  are:  the  ability  to  release 
weaponeering  information  to  coalition  partners;  connectivity 
between  tools  and  mission  planning  systems;  current  weapon 
and  fuze  information;  training  materials,  quick  weaponeering 
guides,  and  graphical  user  interface  enhancements;  and  improved 
blast/ fragment  methodologies  in  support  of  small  precision 
munitions. 


JTCG/ME  continued  to  chair  OUWGs  to  establish  warfighter 
requirements  for  ongoing  development  of  the  JWS  software 
and  DCiDE  tool.  Representatives  from  USCENTCOM, 
USAFRICOM,  USSTRATCOM,  U.S.  Pacific  Command 
(USPACOM),  U.S.  Special  Operations  Command  (USSOCOM), 
the  Services,  the  Defense  Intelligence  Agency  (DIA),  the 
Defense  Threat  Reduction  Agency  (DTRA),  the  Fires  Center 
of  Excellence,  Service  School  Houses,  the  Marine  Aviation 
Weapons/Tactics  Squadron,  Operations  Support  Squadrons, 
Intelligence  Squadrons,  and  numerous  other  operational  units 
routinely  participate. 

JOINT  AIRCRAFT  SURVIVABILITY  PROGRAM  (JASP) 

The  primary  mission  of  the  Joint  Aircraft  Survivability  Program 
(JASP)  is  to  increase  military  aircraft  combat  effectiveness  in 
current  and  emerging  threat  environments.  This  is  accomplished 
through  joint  and  Service  staff  coordination  of  research  and 
development  of  aircraft  survivability  technologies,  assessment 
methodologies,  and  combat  data  collection  to  support  technology 
development  and  acquisition  planning.  In  FY15,  JASP  funded 
55  multi-year  projects  and  delivered  33  final  reports  that  focused 
on  two  major  areas:  aircraft  survivability  technology  research 
and  development  and  aircraft  survivability  assessment  methods. 
JASP  also  investigated  and  catalogued  combat  damage  incidents 
through  the  Joint  Combat  Assessment  Team  (JCAT). 

Aircraft  Survivability  Technology  Research  and  Development 
JASP  has  focused  the  research  and  development  on  three  major 
aircraft  survivability  technology  areas  to:  (1)  help  defeat  the 
near-peer  and  second-tier  adversary  threats  by  developing 
measures  to  avoid  detection  and  engagement  of  advanced  radio 
frequency-guided  and  infrared-guided  threats;  (2)  improve 
aircraft  force  protection;  and  (3)  improve  aircraft  survivability  to 
combat-induced  fires. 

Defeat  Near-Peer  and  Second-Tier  Adversary  Threats. 

To  advance  U.S.  air  superiority  and  improve  U.S.  aircraft 
survivability  against  near-peer  and  second-tier  adversaries,  JASP 
focused  on  addressing  feasible  technologies  and  technology 
improvements  that  would  effectively  counter  prevalent,  current, 
and  emerging  threats. 

•  In  FY15,  JASP  assessed  measures  to  counter  adversary 
radio  frequency-guided  threats  and  anti-access/area-denial 
capabilities.  JASP  funded  the  Naval  Research  Laboratory 
(NRL)  to  significantly  advance  electronic  attack  capabilities 
while  continuing  to  support  radio  frequency-guided  threat 
countermeasure  jamming  development: 

-  Completed  the  development  of  algorithms  for  use  on 
current  and  future  jammers. 

-  Demonstrated  the  potential  of  improved  jammer 
techniques.  The  project  tested  these  in  the  laboratory  and 
will  be  flight  tested  in  FY16. 

•  Many  aircraft  are  equipped  with  active  infrared  jammers, 
flare  dispensers,  and  missile  approach  warning  systems  to 
cue  countermeasures  deployment  (Figure  3).  These  helped 
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reduce  the 
helicopter 
loss  rates 
during  the 
invasion  in 
Iraq  and  the 
subsequent 
counter¬ 
insurgency 
campaign. 

In  FY15, 

JASP 

assessed  measures  to  counter  existing  and  emerging  infrared 
homing  threats  and  focused  on  identifying  solutions  to  counter 
newer  seeker  technologies: 

-  Matured  a  technology  and  technique  to  identify 
infrared-guided  threats  before  or  immediately  after  launch, 
thereby  improving  both  the  timeliness  and  effectiveness  of 
countermeasures . 

-  Analyzed  and  optimized  IRCM  flare  characteristics  that 
affect  the  ejection  velocity  to  improve  countermeasure 
effectiveness.  Ultraviolet  spectral  data  were  gathered  on 
flares  in  flight  test  for  the  first  time.  The  spectrometry  data 
will  be  placed  in  the  Tri-service  Flare  Database  for  the 
Services  to  use  in  expendable  countermeasure  development 
and  assessment.  These  data  will  support  the  development 
of  new  flares  and  evaluate  flare  effectiveness  against 
advanced  EO/IR  guided  threats. 

-  Flight  tested  seven  different  IRCM  flares  to  gather 
trajectory  data  and  high  fidelity  signature  data  to  improve 
the  fidelity  of  tri-service  IRCM  effectiveness  modeling  and 
simulation. 

Aircraft  Force  Protection.  Aircraft  and  crew  losses  due 
to  anti-aircraft  artillery  and  other  unguided  threats  like  the 
rocket-propelled  grenades  (RPGs)  remain  a  concern.  Recent 
helicopter  combat  data  analysis  suggest  that  about  88  percent  of 
the  helicopter  hits  are  due  to  small  arms  and  automatic  weapons 
resulting  in  about  28  percent  of  the  aircraft  losses  and  about 
10  percent  of  the  loss  of  life.  RPG/rockets  account  for  about 
40  percent  loss  of  aircraft  and  about  53  percent  loss  of  life.  To 
improve  the  ability  of  our  aircraft  to  avoid  threat  detection  and 
engagement  and  to  mitigate  damage  and  prevent  destruction 
when  hit,  JASP  has  focused  on  several  schemes  to  protect  U.S. 
aircraft  and  crew  against  these  threats:  (1)  improved  situational 
awareness,  and  (2)  aircraft  system  hardening  solutions. 

•  Situational  Awareness.  Other  than  countermeasures, 
evasive  maneuvers  can  be  effective  in  evading  gunfire  and 
other  unguided  threats,  but  this  is  predicated  on  the  crew’s 
knowledge  of  the  attack,  and  the  location  of  the  attack. 
Acoustic  system  detection  and  smart  mission  management 
systems  that  can  leverage  battlefield  intelligence  databases  are 
a  few  promising  technologies.  JASP  has  been  focusing  on  the 
development  of  an  advanced  hostile  fire  detection  and  shooter 
geo-location  sensor  compatible  with  current  threat  warning 
systems.  JASP  sponsored  development  of  a  sensor  package 


Figure  3.  AH-64  Apache  Helicopter 
Launching  Flares  in  the  Dark 


Figure  4.  Coalition  Helicopter 
Crash  in  Afghanistan 


that  incorporates  both  mid-wave  infrared  (MWIR)  and 
acoustic  waveforms  for  detecting  hostile  fires  and  determining 
the  location  of  the  shooter.  In  FY15  (the  first  year  of  a 
three  year  program),  the  project  developed  a  combined  MWIR 
and  acoustic  detection  and  geo-location  approach  that  enables 
reliable  muzzle  flash  detection,  hostile  intent  determination 
(determination  that  an  aircraft  is  the  intended  target),  and 
reduced  false  alarms  for  a  dependable  shooter  geolocation. 
Aircraft  System  Hardening  Solutions.  Inevitably,  aircraft 
will  be  drawn  into  close  combat  and  will  be  engaged  by 
anti-aircraft  weapons  systems,  the  results  of  which  could 
be  devastating  as  shown  in  Figure  4.  The  majority  of  JASP 
vulnerability 
reduction  focus 
areas  have 
been  aimed  at 
schemes  that 
would  help 
the  aircraft 
system  absorb 
the  damage, 
avoid  aircraft 
destruction, 
and  save  the  crew 
during  an  attack 

or  in  the  event  of  a  crash.  These  include  energy  absorbing 
structure  technologies,  crash  protective  seats,  innovative  armor 
solutions,  and  include  the  assessment  of  aircraft  survivability 
against  unconventional  threats,  e.g.,  directed  energy  lasers. 

-  Initiated  a  project  to  test  hydrodynamic  compliant  structure 
concepts  to  determine  their  application  to  improving 
aircraft  structure  absorbing  technologies.  This  project  is 
scheduled  to  be  completed  in  FY16. 

-  Initiated  a  project  to  test  an  adaptive  seat  energy  absorber 
with  a  goal  of  improving  the  crash  protectiveness  of 
aircraft  seats.  This  project  looks  at  the  effect  of  advanced 
stroking  seat  technology  and  is  scheduled  to  be  completed 
in  FY16. 

-  Developed  a  lighter  more  protective  transparent  armor 
that  defeats  the  7.62x39  ball  round  (single  shot)  at  reduced 
weight  of  4.8  pounds  per  square  foot  (psf)  (a  10  percent 
weight  reduction).  The  project  produced  armors  well 
within  the  expected  design  envelope  and  exceeded 

the  ballistic  goal.  The  success  of  this  project  has  led 
to  a  second  phase  examining  performance  in  different 
environments  and  to  a  more  demanding  threat. 

-  Determined  High  Energy  Laser  effects  against  typical 
aircraft  composite  materials  and  determined  the 
significance  of  vulnerability  factors.  These  results  will 
support  Air  Force  Research  Lab  and  NRL  development  of 
material  solutions. 


Aircraft  Survivability  to  Combat-Induced  Fire. 

Ballistically-induced  fires  are  a  primary  contributor  to  aircraft 
vulnerability.  Understanding  the  causes  and  likelihood  of  fire 
is  a  necessary  part  of  developing  survivable  combat  aircraft  and 
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assessing  their  vulnerability  to  fire.  In  FY15,  JASP  focused  on 
developing  solutions  to  maximize  residual  flight  capability  in  the 
event  of  threat-induced  onboard  fires. 

•  JASP  contributed  to  development  of  a  self-contained  compact, 
low-weight,  low-cost,  high-efficiency  universal  smart 
nozzle  fire  suppression  delivery  system  capable  of  directly 
discharging  multi-phase  agents  to  a  fire  zone.  The  system 
detects  and  locates  a  fire  within  1 00  milliseconds  of  fire 
initiation  (existing  systems  require  2  seconds  to  detect  and 
react  to  a  fire).  The  “smart  nozzle”  fire  suppression  system 
adjusts  the  orientation  and  momentum  of  the  discharging 

fire  suppressant  jet  directly  towards  the  fire  region.  This 
directional  capability  reduces  the  amount  of  agent  required 
while  maintaining  the  ability  to  be  used  multiple  times 
during  the  same  flight.  The  current  weight  of  the  system  is 
approximately  3.5  pounds  with  1.75  ounces  of  agent.  The 
units  would  be  ideal  for  larger  rotary-wing  cabin  areas  where 
troops  reside. 

Aircraft  Survivability  Evaluation  Tools 

•  JASP  continues  to  provide  the  DOD  with  modeling  and 
simulation  capabilities  to  support  Analysis  of  Alternative 
studies  (for  development  and  verification  of  aircraft 
requirements),  to  assess  operational  and  live  fire  test  results, 
and  to  plan  and  rehearse  training  missions.  JASP  focused 
on  increasing  the  capabilities  of  five  survivability-related 
models  with  the  inclusion  of  emerging  threats,  by 
automating  the  analysis  and  post-processing,  and  by 
continuing  to  validate  new  and  existing  model  capabilities: 

(1)  Enhanced  Surface-to-Air  Missile  Simulation  (ESAMS), 

(2)  BRAWLER  (air-to-air  combat),  (3)  Modeling  System  for 
Advanced  Investigation  of  Counter  Measures  (MOSAIC), 

(4)  Computation  of  Vulnerable  Area  Tool  (COVART),  and 

(5)  Fire  Prediction  Model  (FPM)/Next  Generation  Fire  Model. 
In  conjunction  with  the  Defense  Systems  Information  and 
Analysis  Center  (DSIAC),  JASP  has  established  a  standard 
process  for  the  distribution  and  maintenance  of  these  key 
survivability  models. 

•  In  FY15,  DSIAC  distributed  the  latest  versions  of  the 
survivability  models:  BRAWLER  8.1,  COVART  6.5,  ESAMS 
5.0,  and  FPM  4.2. 

-  Studies  supported  by  BRAWLER  in  2015  include:  F-35 
Initial  Operational  Capability  Support  and  Block  4 
capability  studies,  4th  Generation/5 th  Generation  Force 
Mix  Analysis,  F-22  Modernization  Candidate  Selection, 
and  the  F-15  /F-16  Modernization  and  Service  Life 
Extension  Program  (SLEP)  Decisions. 

-  COVART  was  used  to  support  the  F-35  A,  KC-46A, 
AC-130J,  and  CH-53K  programs,  particularly  in  terms  of 
their  design  requirements  and  LFT&E  programs. 

-  FPM  was  also  used  extensively  in  the  KC-46A  LFT&E 
program  for  shot  line  selection  and  pre-test  predictions. 

-  ESAMS  was  used  on  several  studies  for  defining 
requirements  and  operational  concepts  for  upgrades  to 
current  Air  Force  platforms  (F-22A,  F-35,  B-2A)  and 
future  Air  Force  concepts  (Next  Generation  Bomber), 


Flypersonic  Air-breathing  Weapon  Concept  (HAWC), 
and  the  Advanced  Air  Refueling  Capability  Concepts 
(AARCC). 

•  In  FY15,  JASP  funded  three  projects  to  enhance  ESAMS  (used 
for  assessing  radio  frequency-guided  SAM  engagements, 
including  countermeasure  effects  where  possible)  capabilities 
to  respond  to  the  requirements  identified  by  operator  and  user 
needs.  These  enhancements: 

-  Incorporated  an  advanced  naval  SAM  model  developed  at 
the  Office  of  Naval  Intelligence 

-  Updated  the  legacy  model  and  developed  a  new  model  for 
advanced  radio  frequency  chaff 

-  Updated  radar  and  missile  guidance  models  for  two  SAM 
systems  so  they  reflect  the  latest  intelligence  assessment  of 
the  systems  capabilities 

•  JASP  also  supported  the  development  of  high  fidelity  flare 
models  to  design  and  evaluate  flares  against  current  and 
future  EO/IR  guided  threats.  These  projects  developed  the 
requirements  for  high  fidelity  infrared  flare  models  that 
will  work  in  all  the  Services’  hardware-in-the-loop  (HW1L) 
EO/IR  threat  simulation  facilities.  The  DSIAC  distributed 
two  validated  flare  models,  the  pyrotechnic  MJU-32  and 
pyrophoric  MJU-49,  with  the  Flare  Imagery  Analysis  Tool 
vl.O  in  September  2015. 

•  JASP  collaborated  with  Air  Force  Research  Lab  (AFRL) 
and  the  Large  Aircraft  Infrared  Countermeasures  (LAIRCM) 
Program  Office  to  validate  the  MOSAIC  simulation  to 
augment  the  flight  testing  and  HW1L  modeling  supporting 
IOT&E  of  the  LAIRCM  system.  This  capability  will 

also  support  other  future  laser-based  directed  energy 
countermeasure  programs. 

•  Within  the  vulnerability  assessment  technical  area,  JASP 
funded  projects  addressing  the  following  two  major  issues: 

-  Fire  prediction  capability  by  initiating  the  Next  Generation 
Fire  Model  plan,  which  identifies  four  broad  modeling  and 
test  areas  that  need  substantial  improvement:  Penetration 
(particularly  of  fluid  backed  structures),  Energy  Deposition 
(from  fragment  flash  and  armor  piercing  incendiary 
flash),  Fuel  Deposition  (from  hydrodynamic  ram 
phenomenology),  and  Ignition  (the  interaction  between 
Energy  Deposition  and  Fuel  Deposition. 

-  Rotorcraft  crew  casualty  assessment  by  exercising  the 
integrated  Crew  and  Passenger  Survivability  (CAPS) 
methodology.  The  Army  performed  analyses  using 
their  DESCENT  model  to  assess  possible  crash  landing 
conditions  based  on  data  collected  from  the  Navy  Safety 
Center,  the  Combat  Damage  Incident  Reporting  System 
(CDIRS),  and  DSIAC  relevant  to  a  CAPS  assessment 
for  the  CH-53E.  Additionally,  the  Naval  Air  Systems 
Command  will  perform  manned-simulator  tests  to  assess 
what  kind  of  system  malfunctions  may  occur  due  to 
threat  impacts,  and  whether  these  malfunctions  woidd  be 
survivable.  The  project  will  provide  a  better  understanding 
of  the  Integrated  CAPS  methodology  sensitivities  to  help 
prioritize  future  efforts  to  improve  the  modeling  capability, 
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technology  development,  and  rotorcraft  occupant 
survivability. 

Combat  Damage  Assessment 

•  JASP  strengthened  aircraft  combat  damage  incident  reporting 
in  the  Services  and  the  DOD  by  continuing  to  support  the  Joint 
Combat  Assessment  Team  (JCAT).  The  JCAT  is  a  team  of  Air 
Force,  Army,  and  Navy  personnel  that  deploy  to  investigate 
aircraft  combat  damage  in  support  of  combat  operations. 

JCAT  ended  its  operation  in  Afghanistan  in  October  2014 
with  the  return  of  deployed  assessors  to  the  United  States. 

The  team  continued  to  support  assessments  remotely  from 

the  continental  United  States  and  is  ready  to  deploy  rapidly 
outside  of  the  U.S.  if  necessary 

•  In  FY15,  the  JCAT  worked  to  review  and  update  more  than 
10  years  of  combat  damage  reports  in  CD1RS. 

•  The  JCAT  and  Joint  Aircraft  Survivability  Program  Office 
worked  in  coordination  with  the  Office  of  the  Deputy 
Assistant  Secretary  of  Defense  for  Systems  Engineering, 

OSD  (Personnel  and  Readiness),  and  JS/J8  on  an  Aircraft 
Combat  Damage  Reporting  (ACDR)  Doctrine,  Organization, 
Training,  materiel,  Leadership,  Personnel,  Facilities,  and 
Policy(DOTmLPF-P)  Change  Request  (DCR)  proposal 

that  would  institutionalize  ACDR  through  changes  in  joint 
doctrine,  training,  IT  infrastructure,  and  policy.  DOT&E 
approved  the  DCR,  which  was  submitted  to  JS/J8  in 
October  2015. 

•  The  JCAT  trained  the  U.S.  aviation  community  on  potential 
aircraft  threats  and  combat  damage.  This  training  includes 
but  is  not  limited  to:  capabilities  briefs,  intelligence  updates, 
recent  “shoot-down”  briefs  to  discuss  enemy  TTPs,  and  the 
combat  damage  collection  and  reporting  mentioned  above. 

The  attendees  include  aircrews,  maintenance  personnel, 
intelligence  sections,  Service  leadership,  symposia  attendees, 
and  coalition  partners. 

THE  JOINT  LIVE  FIRE  (JLF)  PROGRAM 

In  FY15,  JLF  funded  26  projects  and  delivered  24  reports.  Focus 
areas  for  JLF  included  projects  that  either  1)  characterized  new 
survivability  issues;  2)  characterized  new  lethality  issues;  3) 
improved  accuracy  and  fidelity  of  weapon  data;  4)  improved  test 
methods;  or  5)  improved  modeling  and  simulation  methods. 

Characterization  of  New  Survivability  Issues 

•  The  U.S.  military  operates  numerous  aircraft  powered  by 
the  PT6A  engine.  Variants  of  the  PT6A  engine  are  installed 
on  the  C-12  aircraft  and  other  comparable  platforms  that 
provide  intelligence,  surveillance,  and  reconnaissance; 
medical  evacuation;  and  passenger  and  light  cargo  transport 
for  the  Army,  Navy,  Air  Force,  and  Marine  Corps  in  both 
hostile  and  non-hostile  environments.  In  FY15,  JLF  assessed 
the  survivability  of  these  aircraft  due  to  direct  ballistic 
engagements  to  the  aircraft  propulsion  system. 

-  Despite  PT6A  design  features  such  as  multiple  casings, 
vacuum  pumps,  and  a  centrifugal  impellor  that  inherently 
reduce  ballistic  vulnerability,  the  PT6A  engine  components 


are  vulnerable  to  small  arms  and  fragmenting  threats. 

These  threats  penetrated  the  engine’s  outer  case  at  most 
engine  sections.  Although  none  of  the  resulting  damage 
suggested  a  catastrophic  failure  of  the  engine,  many  of 
the  test  results  would  have  likely  resulted  in  a  major  loss 
of  engine  power  within  five  minutes  or  less.  The  failure 
modes  to  the  engine  include  either  mechanical  damage 
causing  a  loss  of  engine  power  or  mechanical  damage  and 
oil  starvation. 

-  Vulnerability  coidd  be  reduced  further  by  incorporating 
redundant  supply  lines  (fuel  and  oil)  with  smart  valves  in 
the  hot  section  and  shaped  inlet  screens  on  the  oil  pumps, 
and  by  rerouting  critical  lines  to  increase  masking  (to 
select  threats).  Additionally,  ballistic  shielding  could  be 
integrated  with  the  aircraft’s  cowl  structure  to  protect  the 
engine. 

•  The  CH-53E,  CH-47F,  and  CV-22/MV-22  utilize  auxiliary  fuel 
tanks  in  order  to  extend  their  range  and  to  support  forward 
area  refueling  point  (FARP)  missions.  The  FARP  mission 
requires  CH-53E  and  CH-47F  helicopters  to  carry  up  to 
three  800-gallon  auxiliary  fuel  tanks  placed  in  the  aircraft 
cabins.  The  impact  of  these  additional  fuel  tanks  on  aircraft 
survivability  had  to  be  assessed  to  identify  and  address  any 
new  system  vulnerabilities: 

-  JLF  demonstrated  an  increase  in  aircraft  vulnerability 
and  increased  potential  for  fire  from  an  armor  piercing 
incendiary  threat-induced  ullage  explosion  and  dry  bay 
fire  effects  of  the  non-inerted,  non-self-sealing,  800-gallon 
auxiliary  fuel  tanks  in  the  rotorcraft  cabins. 

-  The  project  recommended  design  improvements  to: 

1)  incorporate  technology  (e.g.,  reticulated  foam, 
ullage  inerting  system)  to  mitigate  the  potential  of  fuel 
vapor  ignition  in  the  tank;  and  2)  raise  or  upgrade  the 
fielded  tank’s  self-sealing  performance  to  meet  today’s 
MIL-DTL-27422E  self-sealing  requirement  (leak  reduced 
to  a  damp  seal  within  2  minutes)  for  all  surfaces  on  the 
tank. 

•  Survivability  of  U.S.  ships  to  a  recently  identified  asymmetric 
threat: 

-  In  FY15,  The  Naval  Surface  Warfare  Center,  Carderock 
Division  built  a  full-scale  bomb  using  the  results  of  the 
previously  conducted  quarter  and  half  scale  tests,  along 
with  the  results  of  a  parametric  study  that  varied  different 
threat  weapon  parameters.  The  full-scale  bomb  will  be 
stored,  along  with  supporting  documentation  of  weapon 
design,  and  safe  handling  and  transport,  until  a  suitable 
land  or  at-sea  test  opportunity  becomes  available. 

-  A  full-scale  test  is  needed  to  show  scalability  and  confirm 
the  previously  conducted  tests,  and  to  demonstrate  the 
effects  of  this  threat  against  actual  ship  structure. 

•  Previous  aircraft  LFT&E  assessments  included  limited  or 
no  CAPS  assessments.  JLF  conducted  emergency  egress 
testing  on  a  CFI-47F  helicopter  to  determine  the  time  it  takes 
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occupants  to  exit  the  aircraft  under  a  variety  of  realistic  cabin 
conditions  and  Soldier  combat  gear  loadouts. 

-  Combined  with  the  baseline  fire  assumptions  developed 
under  another  JLF  project,  the  results  provide  an  increased 
understanding  of  the  factors  driving  casualties  during 
aircraft  emergency  egress  for  use  in  future  LFT&E  efforts. 

-  Supplementing  data  available  from  the  Federal  Aviation 
Administration  for  commercial  airlines,  this  project  fills 
a  data  void  for  military  helicopters  with  cabin  fuel  lines 
and  unique  Soldier  equipment  configurations.  This  data 
set  baselines  assumptions  for  the  CH-47  in  particular,  and 
medium-sized  cargo/troop  class  aircraft  in  general  for  use 
in  CAPS  assessments. 


Characterization  of  New  Lethality  Issues 

•  Assessment  of  HELLF1RE  missile  lethality  against  fast-attack 
craft,  a  new  threat  to  U.S.  ships  operating  in  the  littoral 
environment. 

-  JLF  collected  critical  blast,  fragmentation,  and  impact 
damage  data  to  assess  the  capability  of  the  HELLFIRE 
against  the  threat  posed  by  fast-attack  craft  or  other  small 
boats  to  U.S.  ships  operating  in  the  littoral  environment 
(Figure  5). 

-  The  project  demonstrated  the  utility  of  the  unique 
rocket-on-a-rope  testing  technique  to  propel  the  missile 
to  the  target  (a  decommissioned  Coast  Guard  target  craft) 
in  achieving  the  desired  dive  angle,  impact  velocity,  and 
impact  hit  point. 


Weapons  Data  Accuracy 

•  JLF  obtained  new  arena  test  data  on  the  HELLFIRE  missile  to 
permit  improved  collateral  damage  estimates,  risk  to  personnel 
estimates,  and  lethality  effectiveness  of  material  targets  for  the 
HELLFIRE  missile. 

-  The 
HELLFIRE 
missile  is 
frequently 
used  by  the 
warfighters 
to  pursue 
high  value 
targets.  Given 
the  location  of  these  targets  and  the  need  to  minimize 
collateral  damage,  additional  test  data  increased  the 
required  accuracy  and  fidelity  of  the  missile  zonal  data 
file.  The  added  confidence  for  the  HELLFIRE  zonal  data 
will  allow  the  operational  users  to  plan  their  missions 
and  minimize  collateral  damage  with  higher  confidence. 
Blast  pressure  data  and  shape  charge  penetration  data 
were  collected  to  more  accurately  model  the  blast  and 
penetration  capabilities  to  evaluate  the  effectiveness 
against  material  and  personnel  targets. 

-  Fragmentation  files  are  a  critical  input  to  weapon 
effectiveness  analysis  programs  that  are  utilized  in  JMEMs 


Figure  5.  HELLFIRE  against  Fast-Attack  Craft 


to  provide  the  operational  users  with  data  that  can  be  used 
for  mission  planning  purposes. 

•  JLF  was  resourced  to  obtain  new  arena  test  data  on  the  MK 
84  due  to  concerns  about  the  quality  of  the  existing  MK  84 
characterization  data.  JTCG/ME  will  incorporate  the  results  of 
this  test  into  JTCG/ME  products  and  tools. 

-  Initial  examination  of  the  test  data  indicated  a  variance 
from  the  current  characterization  data.  This  variance  has 
a  strong  potential  to  influence  weapon  usage  for  lethality, 
collateral  damage  estimates,  and  risk  assessment. 

-  Data  will  be  compared  with  the  output  of  shock  physics 
predictive  tools  to  improve  the  warhead  detonation 
model  to  produce  high  fidelity  results,  potentially  reduce 
the  number  of  tests  required  for  characterization,  and  to 
provide  a  better  understanding  of  the  fragment  cloud. 

Improvements  of  Live  Fire  Test  Methods 

•  JLF  investigated  the  feasibility  of  using  a  novel  state-of-the 
art  stereographic  video  technology  to  more  efficiently  and 
accurately  characterize  munition  lethality. 

-  Enables  the  collection  of  full  hemispherical  fragmentation 
data,  individual  fragment  characterization,  and  rapid 
post-test  analysis  capability.  This  technology  will  also 
provide  a  vital  method  for  collecting  data  for  future 
weapon  technology,  specifically  directional/focus 
munitions. 

-  This  technology  was  demonstrated  on  an  MK  84  arena 
test.  Utilizing  advanced  image  processing  techniques 
and  a  fragment  tracking  algorithm,  tests  were  able  to 
successfully  demonstrate  the  identification  and  tracking  of 
1,016  fragments  with  stereographic  video  during  the  arena 
test  event. 

•  JLF  investigated  test  instrumentation  to  more  accurately 
capture  and  evaluate  the  blast  effects  on  armored  vehicles 
subjected  to  live-fire  mine  and  IED  tests,  and  improve  the 
ability  to  make  test-to-test  and  test-to-simulation  comparisons. 

-  Legacy  gauges  used  during  these  tests  do  not  permit 
collection  of  the  broader  range  of  data  needed  to  support 
additional  analysis  capabilities  currently  utilized. 

-  High  fidelity  computer  modeling  is  more  commonly 
used  in  vehicle  design  and  test  planning,  so  correlating 
simulation  and  test  data  has  become  increasingly 
important. 

-  This  project  identified  gauge  and  mount  combinations 
with  capabilities  more  appropriately  matched  for  the  blast 
environment.  In  addition  to  providing  data  that  supports 
best  practices  for  gauge  selection  (data  that  describes 
instrumentation  behavior  in  different  frequency  domains 
and,  therefore,  in  different  applications  like  hull  response, 
local  floor  response,  and  seat  input  and  response),  the 
project  also  developed  a  test  protocol  to  allow  for  the 
evaluation  of  new  accelerometers  and/or  isolators  being 
considered  for  use  in  LFT&E. 

•  JLF  investigated  test  instrumentation  to  provide  data  for 
real-time  crew  kill  assessments  in  maritime  live  fire  test 
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events;  more  specifically  a  capability  to  remotely  sense  valid 
fragment  strikes  to  pre-defined  anatomical  regions  strategically 
located  on  a  ballistic  mannequin  that  provides  for  a  limited 
case  real-time  assessment  of  rapid  incapacitation. 

-  JLF  continued  the  development  and  demonstration  of 
the  concept  of  a  hit  sensor  behind  a  calibrated  plywood 
velocity  barrier  that  successfully  detects  a  valid  fragment 
strike  at  a  given  velocity  and  provides  proper  positive 
indication  of  the  hit  to  a  remote  station  and  display. 

-  During  FY14  and  FY15,  JLF  identified  the  fragment 
group  of  interest,  relevant  crew  work  assignments,  critical 
body  regions,  and  developed  the  physical  barrier.  Sensor 
component  development  and  testing  of  both  the  barrier  and 
sensor  will  continue  into  next  year. 

Improvements  of  Live  Fire  Modeling  and  Simulation 

•  Fire  is  the  largest  contributor  to  the  vulnerability  of  fixed-wing 
aircraft  and  a  JLF  project  was  used  to  support  the  fire 
prediction  model  capability  improvements. 

-  Test  data  determined  fire  initiation  as  a  function  of  threat 
type,  velocity,  panel  thickness,  obliquity  angle,  and  fuel 
temperature.  This  is  part  of  a  collaborative  effort  to 
develop  the  ignition  module  for  the  next  generation  fire 
prediction  model. 

•  JLF  provided  a  complete  and  traceable  data  set  to  validate 
blast/ fragmentation  predictions  to  support  the  Endgame 
Manager  model  development.  When  validated,  this  model  can 
be  applied  to  dynamic  aircraft  and  missile  engagements  as  part 
of  an  LFT&E  program. 

-  A  WDU-17/B  warhead  and  a  warhead  of  a  foreign 
surface-to-air  missile  were  successfully  tested  against  a 
full-up  non-operating  F-16  aircraft. 

-  The  JCAT,  in  conjunction  with  Air  Force  engineers, 
conducted  a  post-test  damage  assessment  and  validated 
key  predictions  and  assumptions  for  current  vulnerability 
assessment  models  (i.e.  COVART). 

-  This  testing  proved  out  a  laser  metrology  data  collection 
method  to  allow  for  accurate  three-dimensional  mapping  of 
damage. 

•  JLF  supported  the  improvement  of  the  Behind  Armor  Debris 
(BAD)  algorithm  by  collecting  unprecedented,  high-speed 
images  of  shape-charge  warhead  BAD  using  the  pulsed  laser 
illumination  system  (Figure  6). 

-  Three-dimensional  analyses  of  these  images  produced 
fragment  speeds  as  a  function  of  the  fragment’s  angle  from 
the  residual  jet. 

-  Using  the  velocity  field  based  on  test  data  builds 
confidence  in  the  modeling  of  the  damage  that  BAD 
fragments  cause  to  internal  vehicle  components,  including 
personnel. 

•  The  modeling  and  simulation  challenges  of  weapons  include 
not  only  the  primary  (blast  and  fragmentation)  effects  against 
urban  structures,  but  also  the  potential  for  additional  damage 
and  casualties  created  by  the  failing  structures. 

-  Analytic  tools  for  predicting  weapon  effects  against 
structures  were  originally  created  to  provide  a  conservative 


Figure  6.  High-speed  Image  of  BAD  Fragments 


estimate  of  large  (greater  than  500  pounds)  weapon  effects 
against  military  structures. 

-  Trends  in  weapon  employment  to  smaller  munitions  to 
reduce  collateral  damage  and  increased  use  in  urban 
environments  have  shown  that  the  tools  are  inadequate  in 
these  regimes. 

-  This  shortfall  resulted  in  joint  Army/Air  Force  programs  to 
more  accurately  determine  the  damaging  effects  of  smaller 
munitions  against  urban  structures  and  an  international 
project  agreement  led  by  the  Army  Research  Laboratory 
(ARL)  to  improve  U.S.  predictive  capabilities  in  this  area. 

-  The  Air  Force  Research  Lab  (AFRL)  has  managed 
development  of  a  number  of  fast  running  structural 
response  models  for  smaller  weapon  effects,  but  lacks 
the  validation  data  needed  to  increase  the  confidence  in 
the  models  and  allow  for  their  timely  transition  into  the 
warfighter’s  weaponeering  tool  (JWS). 

-  JLF  project  helped  fund  six  live  fire  experiments  against 
urban  wall  constructions  to  measure  their  response  to 
the  detonation  and  the  potential  damaging  effects  of  the 
resulting  debris  against  personnel  and  equipment.  The 
test  data  will  be  used  to  provide  improved  estimates  of 
weapons  effects. 

•  JLF  continued  a  joint  effort  with  Germany  to  develop 
and  validate  the  Dynamic  Systems  Mechanical  Advanced 
Simulation  (DYSMAS)  hydrocode  used  to  model  bottom  and 
near-bottom  underwater  explosions  effects. 

-  In  FY14,  several  tests  were  conducted  in  the  Briar  Point 
test  pond  at  the  Aberdeen  Test  Center,  Maryland,  using  a 
floating  shock  platform  to  collect  data  on  platform  response 
from  charges  located  at  mid-depth,  near-bottom,  and  on  the 
bottom. 

-  The  analysis  of  those  test  results  was  completed  in  FY15, 
providing  additional  validation  for  the  use  of  DYSMAS  in 
vulnerability  assessments  for  the  modeling  of  underwater 
explosion  loading  and  ship  responses  in  littoral  or  harbor 
environments,  where  bottomed  or  tethered  mines  are  likely 
to  be  encountered. 

•  JLF  supported  the  development  of  shaped  charge  jets  effects 
model. 
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-  Initiation  of  stowed  25  mm  ammunition  is  one  of  several 
lethal  mechanisms  that  can  impart  catastrophic  levels  of 
damage  to  a  ground  vehicle.  Testing  on  stowed  25  mm 
training  rounds  with  shaped-charge  jets  of  varying  size 
and  velocity  collected  quasi-static  pressure  versus  time 
data  that  will  be  used  to  develop  a  new  ammunition 
compartment  vulnerability  model. 

LFT&E  SPECIAL  INTEREST  PROGRAM 

Warrior  Injury  Assessment  Manikin  (WIAMan) 

•  The  Army  will  transition  the  WIAMan  project  to  a  program 
of  record  (POR)  at  Milestone  B,  projected  to  occur  in  FY18. 
The  Program  Executive  Office  for  Simulation,  Training, 
and  Instrumentation  (PEO  STR1)  will  be  the  post-transition 
program  manager.  To  ensure  the  injury  biomechanics, 
anthropomorphic  test  device  (ATD)  expertise,  and  live 

fire  testing  experience  of  the  WIAMan  Engineering  Office 
(WEO)  remains  a  part  of  the  project  post-transition,  the 
Army  Research,  Development,  and  Engineering  Command 
(RDECOM)  and  PEO  STRI  are  in  the  process  of  staffing 
a  Memorandum  of  Agreement  codifying  the  roles  and 
responsibilities  of  their  respective  offices  post-transition. 

•  In  preparation  for  establishing  the  POR,  the  Army  initiated  an 
effort  to  create  a  Test  Capability  Requirements  Document  for 
the  WIAMan  system,  which  documents  the  key  perfonnance 
parameters  and  key  system  attributes  the  system  is  required 
to  meet  in  order  to  progress  through  its  milestones.  This 
document  will  be  signed  by  DOT&E,  RDECOM,  and  the 
Army  Test  and  Evaluation  Command,  and  will  include 
requirements  to  execute  exploratory  research  on  the  response 
of  females  to  underbody  blast  loading  conditions  to  determine 
the  scope  a  test  program  that  would  be  required  for  the 
development  of  a  female  ATD. 

•  The  WEO  continued  to  demonstrate  that  the  current  ATD  used 
in  LFT&E,  the  Hybrid  III,  lacks  biofidelity  in  the  underbody 
blast  (UBB)  test  environment,  meaning  it  does  not  respond  as 
a  human  does  when  exposed  to  similar  loads.  Establishing  the 
human  response  to  the  UBB  domain  is  essential  in  developing 
a  military-specific  ATD,  and  a  critical  first  step  is  establishing 
biofidelity  response  corridors  (BRCs)  for  the  human  body 
regions  of  interest. 

-  In  FY15,  the  project  delivered  10  of  an  expected  16  total 
BRCs,  with  the  remaining  BRCs  to  be  delivered  by 
3QFY16.  These  BRCs  are  focused  on  the  human  response 
in  the  head/neck,  lumbar  spine,  pelvis,  and  lower  leg/foot 
and  ankle  body  regions.  BRCs  for  the  whole  body  are  also 
under  development. 

-  In  addition,  the  Army  generated  initial  data  on  the  tolerance 
of  bones  to  severe  loading  conditions  and  developed 
human  injury  probability  curves  for  foot  and  ankle 
fractures.  The  investigation  of  these  foot  and  ankle  injuries 
benefitted  from  updated  analyses  of  injuries  experienced 
by  Soldiers  in  combat;  these  analyses  revealed  greater 
detail  on  the  exact  type  and  nature  of  the  skeletal  fractures 
suffered. 


•  In  FY15,  the  WEO  awarded  a  contract  to  Diversified  Technical 
Systems  (DTS)  for  a  technical  demonstrator  of  the  ATD, 
which  DTS  is  expected  to  deliver  at  the  end  of  1QFY16.  The 
WEO  and  DTS  utilized  computational  modeling  to  prescribe 

a  range  of  candidate  materials  for  the  ATD  and  its  parts;  use 
of  the  proper  materials  is  critical  for  establishing  and  ensuring 
the  robustness  of  the  ATD  in  UBB  conditions,  as  well  as  to 
ensure  the  response  of  the  ATD  falls  within  the  BRCs.  DTS 
will  perform  iterative  testing  that  includes  matched  pair 
BRC  testing  in  multiple  phases  to  allow  opportunities  to 
alter  material  selection  and  component  geometry  to  improve 
biofidelity.  The  first  tests  of  WIAMan  ATD  hardware  of  the 
cervical  spine  and  pelvis  were  conducted  at  Duke  University 
and  the  University  of  Virginia,  respectively. 

•  The  WEO  is  developing  an  optimized  ATD  finite  element 
model  (FEM)  as  test  results  become  available.  This  FEM 
will  support  analyses  to  accelerate  the  re-design  of  the  ATD 
to  achieve  strength-of-design,  biofidelity,  and  usability  goals. 
The  FEM  is  also  used  to  produce  pre-test  predictions  to  aid 
in  test  planning  and  identifying  risks  to  the  robustness  of  the 
design  and  the  compliance  of  the  materials.  To  date,  a  full 
three-dimensional  description  of  the  ATD  has  been  created, 
along  with  models  of  the  test  devices  in  which  the  ATD  will  be 
evaluated. 

•  The  technical  achievements  made  by  the  WEO  and  the 
concerted  effort  by  the  Army  to  create  the  foundation  for  a 
formal  acquisition  program  represent  major  steps  forward  for 
the  WIAMan  project,  and  the  effort  is  poised  to  make  addition 
progress  in  FY16  and  beyond.  However,  the  Army  has  not 
programmed  funding  for  the  WIAMan  project  past  FY17. 

The  T&E  Resources  section  of  this  annual  report  provides 
additional  information  regarding  the  funding  history  of  the 
WIAMan  project. 

Home  Made  Explosives  (HMEs) 

•  DOT&E  continues  to  participate  in  Army-led  efforts 
to  characterize  a  surrogate  for  IEDs  often  encountered 
in  Afghanistan  operations,  also  known  as  Home-Made 
Explosives  (HME).  The  ongoing  HME  characterization 
(HME-C)  effort  originated  to  address  concerns  regarding  the 
Department’s  ability  to  test  operationally  significant  scenarios 
involving  underbody  blast  threats,  and  to  ensure  adequate 
LFT&E  of  military  vehicles  now  and  in  the  future.  The 
HME-C  program  intends  to  do  the  following: 

-  Establish  a  threat  surrogate  for  HME,  approved  by  the 
Army’s  Office  of  the  Deputy  Chief  of  Staff,  Intelligence. 

-  Characterize  the  surrogate  threat’s  effect  on  ground  vehicle 
targets  to  determine  its  suitability  for  use  in  Title  10  ground 
vehicle  LFT&E. 

-  Ensure  the  HME  surrogate’s  testability,  repeatability,  and 
measurability. 

-  Conduct  additional  characterization  of  TNT,  which  is  the 
military  bulk  explosive  currently  used  in  LFT&E.  The 
DOD  currently  has  insufficient  data  to  relate  testing  under 
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current  conditions  to  other  operationally  significant  soils, 
test  sites,  and  threats. 

-  Establish  a  new  soil  standard  for  future  LFT&E  that 
includes  buried  underbody  blast  threats. 

-  Investigate  and  develop  new  test  protocols  to  ensure 
adequate  and  repeatable  underbody  blast  threat  testing  in 
the  future. 

•  In  FY15,  the  HME-C  program  completed  multiple  test  phases, 
which  are  intended  to  investigate  the  repeatability  of  HME 
surrogate  effects  relative  to  those  of  TNT  and  the  effects  of  soil 
condition  and  IED  emplacement  on  HME  threat  performance. 
DOT&E  is  currently  working  with  the  Army  to  analyze  the 
data  from  the  HME-C  program  to  support  decisions  regarding 
the  use  of  HME  in  LFT&E  and  the  implementation  of  new  soil 
standards  (soil  type,  condition,  and  preparation)  for  underbody 
blast  threat  testing.  The  Department  expects  to  make  these 
decisions  in  FY16. 

Small  Boat  Shooters  ’  Working  Group 

•  Small  boats  represent  a  growing  threat  class  to  ships  operating 
in  littoral  waters,  and  are  targeted  by  a  wide  variety  of  weapon 
systems. 

-  In  FY15,  DOT&E  sponsored  the  fourth  annual  Small 
Boat  Shooter’s  Working  Group,  which  examined  the 
general  nature  of  the  small  boat  threat  in  littoral  waters; 
summarized  the  threat  classes  and  available  targets  and 
models  available  for  ammunition,  rocket,  and  tactical 
missile  weapon  systems;  and  attempted  to  synchronize 
various  LFT&E  and  other  operational  test  approaches 


among  the  various  programs/Services  by  sharing  the 
breadth  of  test  and  evaluation  options  available  to 
evaluators. 

-  The  working  group  assessed  the  nature  of  the  small  boat 
threat;  the  availability  of  targets  and  lethality  models 
representing  those  threats;  the  data  collection,  test 
techniques,  and  instrumentation  that  have  been  applied  to 
small  boats;  and  the  performance  of  shipboard  and  aircraft 
weapons  against  small  boat  threats. 

-  The  working  group  identified  the  need  for  incorporating 
a  broader  variety  of  surrogate  small  boat  targets  into 
operational  testing,  and  for  gathering  better  live  fire  data 
from  operational  test  events  (e.g.,  HELLFIRE  Romeo 
missiles  fired  against  the  ex-Coast  Guard  CG-41  FAC 
surrogate,  and  HELLFIRE  longbow  missiles  vertically 
fired  from  a  ship  against  High  Speed  Mobile  Surface 
Targets  as  part  of  the  Littoral  Combat  Ship  (LCS) 
program). 

RECOMMENDATION 

1.  The  Army  should  identify  and  secure  the  funding  outlined 
in  the  WIAMan  Program  Office  Estimate  so  the  project  can 
continue  uninterrupted.  Any  disruption  in  funding  is  likely  to 
affect  significantly  the  Army’s  ability  to  execute  the  project 
in  its  entirety.  Any  program  delays  will  force  the  Department 
to  continue  using  inadequate  vehicle  underbody  blast  test 
instrumentation  and  injury  criteria. 
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Cybersecurity 


SUMMARY 


DOT&E  observed  improvements  in  several  cybersecurity 
areas  within  the  Department  of  Defense  (DOD)  this  past  year; 
however,  the  Department’s  warfighting  missions  and  systems 
remain  vulnerable  to  cyber-attack.  Observed  improvements 
include  enhanced  protection  of  some  network  elements,  greater 
challenges  for  cyber  opposing  forces  (OPFOR)  attempting 
to  gain  access  to  networks,  and  greater  awareness  by  DOD 
leadership  of  the  potential  impact  that  cyber  attacks  could  have 
on  key  systems  and  the  critical  missions  they  support.  However, 
these  improvements  are  not  enough  to  ensure  mission  success. 

In  FY15  operational  tests  and  exercise  assessments,  the  cyber 
OPFOR  was  frequently  in  a  position  to  deliver  cyber  effects  that 
could  degrade  the  performance  of  operational  missions.  Exercise 
authorities  seldom  permitted  cyber  attacks  from  being  conducted 
to  the  full  extent  that  an  advanced  adversary  would  likely  employ 
during  conflict,  so  actual  data  on  the  scope  and  duration  of  cyber 
attacks  are  limited.  Additionally,  exercise  authorities  often 
declined  to  allow  kinetic  effects  based  on  data  exfiltrated  by  the 
cyber  OPFOR. 

DOT&E  believes  the  reluctance  by  Combatant  Commands 
(CCMDs)  and  Services  to  permit  realistic  cyber  effects  during 
major  exercises  is  due  to  the  requirement  to  achieve  numerous 
other  training  objectives  in  those  exercises.  Additionally, 
exercise  authorities  have  stated  they  fear  that  cyber  attacks 
could  distract  from — and  possibly  preclude — achieving  these 
objectives.  However,  based  on  the  increasing  frequency  of 
cyber  attacks  throughout  the  world,  CCMDs  should  expect  cyber 
attacks  to  be  present  for  all  critical  missions  they  may  be  ordered 
to  execute.  In  order  to  attain  a  high  state  of  mission  readiness, 
CCMDs  and  supporting  defenders  shoidd  conduct  realistic  tests 
and  training  that  include  cyber  attacks  and  effects  representative 
of  those  that  advanced  nation  states  would  execute. 

Identified  Cyber  Vulnerabilities 

As  in  previous  years,  assessment  teams  consistently  found  four 
categories  of  vulnerabilities  in  both  system  tests  and  exercise 
assessments: 

•  Exposed  or  poorly  managed  credentials 

•  Systems  not  configured  to  identified  standards 

•  Systems  not  patched  for  known  vulnerabilities 

•  System/network  services  and  trust  relationships  that  provide 
avenues  for  cyber  compromise 

Noteworthy  Successes  by  Network  Defenders 

Although  defenses  need  improvement,  there  were  specific 
instances  where  defenses  worked,  including  the  following: 

•  The  cyber  OPFOR  found  that  vulnerabilities  routinely 
available  in  many  networks  were  not  present  in  some  networks 
due  to  timely  upgrades  and  software  patches. 


•  A  layered  approach  to  stop  primary  attack  vectors,  such  as 
phishing,  proved  effective  at  defending  networks  and  forced 
the  cyber  OPFOR  to  expend  more  time  and  deploy  more 
advanced  capabilities.  Layered  defenses  that  occupy  the 
adversary’s  time  away  from  a  target  may  buy  sufficient  time 
for  the  Blue  Force  to  sustain  its  critical  missions. 

•  Application  whitelisting,  where  network  defenders  allow  only 
“known  good”  applications  to  operate  on  a  network,  precluded 
the  cyber  OPFOR  from  expanding  its  foothold  in  the  network. 

•  A  local  hunt  team  supported  by  a  Cyber  Protection  Team 
(CPT)  was  effective  at  log  reviews  that  resulted  in  detection  of 
the  cyber  OPFOR’s  presence. 

Detection  tools  used  by  network  defenders  were  primarily 
signature-based  and  dependent  upon  commercial  tools  adapted 
for  DOD  use.  However,  the  majority  of  adversarial  accesses 
involved  the  use  of  “native”  software  normally  available  within 
the  networks  and  operating  systems.  Since  misuse  of  native 
software  is  less  easily  detected  and  eliminated  than  malware, 
the  DOD  should  augment  current  network  defenses  with 
behavior-based  and  heuristic-type  sensors. 

Cyber  Red  Teams 

The  demand  on  DOD-certified  Red  Teams,  which  are  the  core  of 
the  cyber  OPFOR  teams,  has  increased  significantly  in  the  past 
3  years.  In  the  same  timeframe,  the  private  sector  has  hired  away 
members  of  Red  Teams,  resulting  in  staffing  shortfalls  during  a 
time  when  demand  is  likely  to  continue  to  increase.  This  trend 
must  be  reversed  if  the  DOD  is  to  retain  the  ability  to  effectively 
train  and  assess  DOD  systems  and  Service  members  against 
realistic  cyber  threats. 

Persistent  Cyber  OPFOR  (PCO)  and  Continuous  Assessments 

In  FY15,  U.S.  Pacific  Command  (USPACOM)  leadership 
approved  year-round  activities  of  a  Persistent  Cyber  OPFOR 
(PCO)  in  order  to  portray  a  more  realistic  cyber  adversary  in 
training  and  assessment  events,  and  make  the  most  efficient  use 
of  scarce  Red  Team  personnel.  The  PCO  employs  DOD-certified 
Red  Teams  in  longer-duration  activities  to  be  more  representative 
of  enduring  threat  actors  than  can  be  portrayed  in  a  brief 
exercise  period.  This  PCO  has  already  helped  USPACOM  find 
and  remediate  mission-critical  vulnerabilities  that  might  have 
otherwise  gone  undetected. 

USPACOM  also  agreed  to  a  Theater  Cyber  Readiness  Campaign 
(TCRC)  in  FY15.  The  TCRC  included  more  frequent  cyber 
assessment  activities  and  allowed  USPACOM  to  optimize 
cybersecurity  preparations  in  smaller  events  throughout  the 
year,  and  then  examine  a  larger  array  of  challenges  in  a  capstone 
exercise  event.  U.S.  European  Command  (USEUCOM)  and 
U.S.  Northern  Command  (NORTHCOM)  are  also  developing 
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TCRCs.  In  theaters  where  the  PCO  and  continuous  assessments 
are  active,  DOD  is  better  positioned  to  find  cybersecurity 
problems,  develop  solutions  or  mitigation  strategies,  and  verify 
that  fixes  are  in  place  and  effective. 

Cyber  Protection  Team  (CPT)  Assessments 

In  FY15,  DOT&E  continued  a  partnership  with  U.S.  Cyber 
Command  (USCYBERCOM)  to  experiment  with  evolving  cyber 
range  capabilities  and  the  potential  benefits  of  team-training 
for  representatives  from  CPTs.  Most  participants  stated  the 
opportunity  to  experience  cyber  attacks  as  a  team  on  a  realistic 
range  network  that  included  a  live  OPFOR,  and  then  engage 
with  the  OPFOR  during  after-action  reviews,  constituted  the  best 
training  they  had  received  to  date.  They  also  expressed  a  strong 
desire  for  this  type  of  training  at  their  individual  duty  stations. 
DOT&E  observed  CPTs  with  team  training  performed  better  than 
CPTs  without  team  training,  and  expects  that  significant  time  on 
realistic  ranges  will  be  instrumental  to  CPTs  attaining  an  effective 
operational  capability.  DOT&E  also  observed  some  individuals 
assigned  to  CPTs  do  not  possess  the  proper  training,  background, 
or  motivation  to  become  effective  CPT  members.  DOT&E 
acquired  and  enhanced  survey  tools  that  can  help  determine 
individual  suitability  for  CPTs,  and  has  offered  these  tools  to 
USCYBERCOM  and  Service  cyber  components. 

Cyber  Ranges 

The  FY15  National  Defense  Authorization  Act  (NDAA)  directed 
DOD  to  establish  an  Executive  Agent  (EA)  for  cyber  training 
ranges  and  an  EA  for  cyber  testing  ranges.  DOT&E  has  often 
used  cyber  ranges  for  events  that  combine  testing  and  training. 
Such  combined  events  make  efficient  use  of  scarce  cyber  range 
resources.  The  creation  of  two  separate  EAs — with  separate 
responsibilities  and  incentives — would  make  it  difficult  to 
conduct  combined  activities  in  a  timely  manner.  A  single  EA 
should  be  designated  with  the  authority  to  oversee  funding  and 
personnel  for  all  DOD-owned  cyber  ranges. 

Conclusions 

During  exercises,  DOD  network  defenders  continue  to 
demonstrate  low  detection  rates  of  cyber  OPFOR  activities.  The 
Microsoft  Corporation  has  recently  adopted  the  assumption 
that  all  systems  are  compromised  (“Assume  Breach”),  which  is 
an  appropriate  posture  for  the  Department  as  well.  The  DOD 
should  experiment  with  and  perfect — with  rigorous  test  and 
evaluation — the  tools,  tactics,  and  operational  procedures  that  can 
quickly  identify  and  stop  ongoing  cyber  attacks. 

Combatant  Commands  should  make  serious  preparations  to 
conduct  all  critical  missions  in  a  cyber-contested  environment, 
and  perform  periodic  operational  demonstrations  to  ensure 
mission  success.  These  demonstrations  should  include 
operational  units,  all  network  defenders,  and  CPT  elements 
that  would  be  expected  in  support  of  each  mission.  DOT&E  is 
prepared  to  provide  support  to  plan,  conduct,  and  evaluate  such 
demonstrations  on  both  operational  networks  and  in  appropriate 
cyber  range  environments. 


Recommendations 

DOT&E  recommends  the  CCMDs  and  Services: 

•  Demonstrate  the  ability  to  sustain  critical  missions  in  a 
contested  cyber  environment,  consistent  with  Secretary  of 
Defense  and  Chairman,  Joint  Chiefs  of  Staff  guidance.1 

•  Develop  tools,  tactics,  and  operational  procedures  and 
perform  regular  battle  drills  with  playbooks  to  ensure  mission 
accomplishment  in  the  contested  cyber  environment. 

•  Allow  threat-representative  cyber  effects,  using  a  persistent 
cyber  OPFOR,  during  all  major  exercises. 

•  Request  the  leadership  of  the  DOD  Enterprise  Cyber 
Range  Environment  and  programs  of  record  create  range 
environments  to  support  the  demonstration  of  cyber  effects 
that  are  not  suitable  for  operational  networks,  and  the 
development  and  testing  of  remediation  options  for  cyber 
vulnerabilities. 

DOT&E  recommends  the  DOD: 

•  Accelerate  the  implementation  of  key,  cybersecurity  best 
practices  to  include  application  whitelisting;  secure  system 
configurations;  and  rapid  patch  application. 

•  Reduce  the  number  of  users  with  administrative  privileges. 

•  Increase  cybersecurity  training  and  accountability  for  all 
personnel  who  use  DOD  networks. 

•  Designate  a  single  EA  for  cyber  ranges  with  the  authority 
to  oversee  funding  and  personnel  for  all  DOD-owned  cyber 
ranges. 

•  Develop  options  to  attract  and  retain  experienced  cybersecurity 
personnel,  especially  personnel  with  Red  Team  and  cyber  test 
experience. 

DOT&E  recommends  DOD  network  defenders  implement  the 

following  critical  cybersecurity  measures: 

•  Limit  the  availability  of  native  administrative  tools  that 
adversaries  can  exploit  to  only  key  personnel. 

•  Limit  access  to  password  and  operational  data  only  to 
authorized  users  with  need-to-know. 

•  Increase  network  segmentation  and  remote  authentication 
policies  to  create  a  layered  defense  of  critical  assets. 

•  Deploy  heuristic  and  behavior-based  intrusion-detection 
systems  and  procedures  to  assist  in  the  identification  of 
suspicious  network  and  system  activity. 

DOT&E  recommends  the  Services  and  EAs  for  the  DOD  cyber 

ranges: 

•  Provide  all  CPTs  with  ready  access  to  range-network 
environments  for  routine  training  and  tactics  development. 

•  Employ  survey  and  other  testing  means  to  identify  candidates 
for  the  Cyber  Mission  Force  and  to  determine  their  readiness 
to  move  into  advanced  training  and  mission  status. 


For  example,  the  DOD  Cyber  Strategy  dated  April  2015,  and  the  DOD 
Cybersecurity  Culture  and  Compliance  Initiative  memorandum,  signed  by 
the  Secretary  of  Defense  and  the  Chairman  of  the  Joint  Chiefs  of  Staff,  dated 
September  30,  2015,  and  agency  cyber  commands),  and  Tier  1  (DOD-wide, 
e.g.,  U.S.  Cyber  Command) 
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DOT&E  recommends  that  the  Undersecretary  of  Defense 
for  Acquisition,  Technology  and  Logistics,  require  programs 
of  record  to  demonstrate  they  have  no  critical  cybersecurity 
vulnerabilities  prior  to  proceeding  to  the  next  acquisition 
milestone  and  prior  to  fielding. 


DOT&E  recommends  the  Chief  Information  Officer  (CIO) 
update  DOD  Instruction  8330.01  (Interoperability  of  Information 
Technology)  to  require  performance  of  a  cybersecurity  risk 
assessment  prior  to  connecting  systems  or  networks  for 
interoperability  reasons. 


FY1 5  ACTIVITIES  AND  OBSERVATIONS 


DOT&E  conducted  33  cybersecurity  operational  tests  of 
acquisition  programs  and  13  assessments  during  CCMD  and 
Service  training  exercises,  as  shown  in  Table  1 . 

Cybersecurity  OT&E  of  Acquisition  Programs 

Cybersecurity  operational  testing  has  two  phases,  as  prescribed 
by  DOT&E  in  August  2014: 

•  Cooperative  Vulnerability  and  Penetration  Assessments 
(CVPA).  Operational  test  agencies  conduct  overt  and 
comprehensive  vulnerability  and  penetration  assessments 
in  cooperation  with  the  acquisition  program  manager  to 
characterize  the  cybersecurity  status  of  a  system.  CVPAs 
include  all  major  system  interfaces  and  operational 
environments. 

•  Adversarial  Assessments  (AA).  Operational  test  agencies 
conduct  AAs  to  determine  the  operational  impact  of  system 
cyber  vulnerabilities.  AAs  evaluate  the  ability  of  a  unit 
equipped  with  the  system  to  conduct  assigned  missions  in  the 
expected  operational  environment  in  the  presence  of  a  realistic 
cyber  threat.  The  operational  environment  includes  the  local 
and  higher-echelon  cyber  defenders  that  support  the  system 
under  test  during  its  mission. 

In  FY15,  DOT&E  reviewed  and  provided  cybersecurity  test 
input  for  105  Service  and  DOD  systems,  including  65  Test  and 
Evaluation  Master  Plans  and  40  operational  test  plans. 

The  four  common  cybersecurity  shortfalls  found  during  tests  of 
acquisition  systems  were: 

•  Exposed  or  poorly  managed  credentials 

•  Systems  not  configured  to  identified  standards 

•  Systems  not  patched  for  known  vulnerabilities 

•  System/network  services  and  trust  relationships  that  provide 
avenues  for  cyber  compromise 

The  types  of  systems  at  risk  from  cyber  threats  include 
non-Internet  Protocol  networks  such  as  the  1553  and  Controller 
Area  Network  data  buses.  A  number  of  programs  incorporate 
sensitive  industrial  control  systems  and  programmable  logic 
controllers,  or  deploy  capabilities  on  commercial  clouds.  The 
diversity  of  systems  and  services  susceptible  to  cyber  attack  will 
require  new  test  capabilities  and  environments  for  networks  at  all 
levels  of  security  classification. 

In  order  to  plan  and  conduct  adequate  OT&E  of  these  types  of 
systems  and  networks,  test  teams  will  require  in-depth  knowledge 
about  their  operations  and  unique  vulnerabilities.  As  the  Services 


begin  to  use  commercial  cloud  services  for  data  storage,  it  is 
critical  that  DOD  develop  contracts,  policies,  and  regulations  that 
permit  independent  DOD  cybersecurity  testing  of  commercial 
services  and  sites. 

Cybersecurity  Assessments  during  CCMD  and  Service 
Exercises 

DOT&E’s  Cybersecurity  Assessment  Program  observes  and 
reports  on  DOD  efforts  to  improve  cybersecurity  and  cyber 
functionality  through  assessments  of  the  CCMDs  and  Services. 
With  DOT&E  oversight,  the  five  DOD  Operational  Test  Agencies 
and  the  Standing  Test,  Assessment,  and  Rehearsal  Team  (START) 
completed  cybersecurity  assessments  during  eight  CCMD 
exercises,  two  Service  exercises,  and  three  assessments  of 
operational  sites.  The  START  is  the  inclusive  term  for  DOT&E 
partnerships  with  organizations  and  individuals  who  possess 
unique  skills  and  experience  across  the  cybersecurity,  cyber 
range,  and  operational  test  domains.  DOT&E  used  the  START  in 
FY15  to  plan  and  conduct  cyber  assessments  in  USPACOM,  to 
jump-start  the  testing  of  programmable  logic  controllers,  to  plan 
and  conduct  tests  of  offensive  cyber  capabilities,  and  develop  and 
conduct  cyber  range  events. 

To  ensure  operational  realism  and  standardization  of  assessments, 
in  FY15,  DOT&E  also  published  an  Assessment  Handbook 
that  outlines  procedures;  identifies  required  data  elements;  and 
states  expectations  for  the  planning,  conduct,  and  reporting  of 
cybersecurity  assessments. 

Cyber  Assessment  Master  Plan  (CAMP) 

The  Cyber  Assessment  Master  Plan  (CAMP)  is  a  3-year  plan 
that  identifies  a  CCMD’s  priority  missions  and  specifies  when 
the  CCMD  plans  to  assess  those  missions  in  a  contested  cyber 
environment.  CAMPs  are  signed  by  CCMD  and  DOT&E 
leadership  to  focus  resources  and  planning  of  assessments  that 
will  meet  the  requirements  of  the  DOD  cyber  strategy.  For  each 
mission  identified  in  the  CAMP,  DOT&E  will  plan  a  TCRC 
that  will  include  multiple  building  block  events  that  lead  to  a 
stressing  capstone  event  for  the  mission  to  be  assessed.  A  TCRC 
may  span  multiple  years  until  the  CCMD  has  demonstrated  the 
TCRC  mission  will  be  effective  when  stressed  by  an  advanced 
cyber  adversary,  and  that  key  supporting  networks  and  systems 
are  sufficiently  secure  or  resilient.  In  FY15,  DOT&E  began 
development  of  CAMPs  with  USPACOM,  USEUCOM,  and 
USNORTHCOM. 
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TABLE  1.  CYBERSECURITY  OPERATIONAL  TESTS  AND  ASSESSMENTS  IN  FY15 

EVENT 

TYPE 

SYSTEM  OR  EXERCISE  AUTHORITY 

DOD  Automated  Biometric  Information  System 

F-35  Lightning  II  Joint  Strike  Fighter 

Aegis  Weapons  System 

Integrated  Personnel  and  Pay  System  -  Army 

Aegis  Ballistic  Missile  Defense 

Joint  Warning  and  Reporting  Network 

Air  Force  Distributed  Common  Ground  System 

KC-46  Pegasus  -  Tanker  Replacement  Program 

Consolidated  Afloat  Network  and  Enterprise  Services 

Littoral  Combat  Ship 

Defense  Agencies  Initiative 

Logistics  Modernization  Program 

Distributed  Common  Ground  System  -  Army 

Mid-Tier  Networking  Vehicular  Radio 

Defense  Enterprise  Accounting  and  Management  System 

MV-22  Osprey  -  Joint  Advanced  Vertical  Lift  Aircraft 

CVPA  and 

Defense  Medical  Information  Exchange 

Pueblo  Chemical-Agent  Destruction  Pilot  Plant 

AA 

Department  of  Navy  Large  Aircraft  Infrared  Countermeasures 

XM1 156  Precision  Guidance  Kit 

Defense  Readiness  Reporting  System 

AN/TPQ-53  Radar  System 

F-22  -  RAPTOR  Advanced  Tactical  Fighter 

Surface  Electronic  Warfare  Improvement  Program 

Global  Combat  Support  System  -  Army 

RQ-21 A  Small  Tactical  Unmanned  Aerial  System 

Global  Combat  Support  System  -  Joint 

Surveillance  Towed  Array  Sensor  System  Low  Frequency  Active 

Guided  Multiple  Launch  Rocket  System  -  Alternative  Warhead 

Theater  Medical  Information  Program  -Joint 

MQ-1 C  Gray  Eagle  Unmanned  Aircraft  System 

Space  Classified  Program 

Integrated  Electronic  Health  Record 

U.S.  Africa  Command  Judicious  Response  201 5 

U.S.  Special  Operations  Command  Tempest  Wind  2015 

Exercise 

Assessment 

U.S.  Northern  Command  Vigilant  Shield  201 5 

U.S.  Strategic  Command  Global  Lightning  2015 

U.S.  European  Command  Austere  Challenge  201 5 

U.S.  Transportation  Command  Turbo  Challenge  2015 

U.S.  Pacific  Command  Pacific  Sentry  201 5 

U.S.  Army  Warfighter  201 5-4 

U.S.  Southern  Command  Integrated  Advance  2015 

U.S.  Navy  USS  Harry  S.  Truman  Sustainment  Exercise 

Site 

U.S.  Central  Command  Air  Forces  Central  Command 

U.S.  Forces  Korea  Headquarters  and  Osan  Air  Base 

Assessment 

U.S.  Air  Force  61 3  Air  Operations  Center 

CVPA  -  Cooperative  Vulnerability  and  Penetration  Assessment;  AA  -  Adversarial  Assessment 

Cyber  Blue  and  Red  Teams 

DOD  cyber  teams  include  organizations  that  provide  OPFOR 
aggressors  (Red  Teams)  as  well  as  penetration  testers  and  teams 
that  perform  other  cybersecurity  assessments  (Blue  Teams). 
DOT&E  guidance  establishes  data  and  reporting  requirements  for 
cyber  team  involvement  in  both  operational  tests  of  acquisition 
systems  and  exercise  assessments. 

The  demand  on  DOD-certified  Red  Teams,  which  are  the  core  of 
the  cyber  OPFOR  teams,  has  increased  significantly  in  the  past 
3  years.  In  the  same  timeframe,  the  Cyber  Mission  Force  and 
private  sector  have  hired  away  members  of  Red  Teams,  resulting 
in  staffing  shortfalls  at  a  time  when  demand  is  likely  to  continue 
to  increase.  This  trend  must  be  reversed  if  the  DOD  is  to  retain 
the  ability  to  effectively  train  and  assess  DOD  systems  and 
Service  members  against  realistic  cyber  threats. 

In  FY15,  the  almost  non-stop  pace  of  events  for  all  cyber 
teams  challenged  their  ability  to  provide  complete  data  sets 
and  complete  reports.  Without  these  data  and  reports,  network 
defenders  and  trainers  will  not  have  the  critical  inputs  they  need 
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to  develop  effective  mitigations  or  perform  effective  training  on 
new  procedures.  DOT&E  worked  with  the  Cyber  Red  Teams  to 
improve  data  collection  and  reporting  efforts,  and  is  examining 
new  capabilities  such  as  graphical  free-form  databases  and 
automated  collection  tools  intended  to  reduce  the  burden  on  the 
teams  while  providing  the  required  information  for  analysis. 

Persistent  Cyber  Opposing  Force  (PCO) 

Red  Teams  or  cyber  OPFOR  require  authority,  typically  called 
“ground  rules”  or  “rules  of  engagement,”  to  operate  on  DOD 
networks  and  systems  for  operational  tests  and  training  exercises. 
The  creation  and  staffing  of  separate  ground  rules  for  each  event, 
network,  and  participating  cyber  Red  Team  is  an  administrative 
burden  that  has  delayed  cybersecurity  operational  tests  and  the 
start  of  activities  in  support  of  training  exercises.  The  PCO 
is  intended  to  help  overcome  these  problems  and  enhance 
cybersecurity  assessments. 

The  PCO  is  the  DOT&E-sponsored  collection  of  Cyber 
Red  Teams  that  perform  long-duration  adversarial 
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activities  in  approved  CCMD  and  Service  theaters.  These 
threat-representative  activities  are  designed  to  make  more 
efficient  use  of  Red  Team  personnel,  provide  more  realistic 
cybersecurity  assessment  opportunities  throughout  the  year, 
and  provide  better  training  opportunities  for  the  CCMDs  and 
the  Cyber  Mission  Force.  DOT&E  believes  that  such  training 
throughout  the  year  will  improve  CCMD  defensive  and  offensive 
cyberspace  operations  and  readiness  to  conduct  critical  missions. 

The  U.S.  Army  Threat  Systems  Management  Office  provides 
day-to-day  management  of  PCO  activities  and  helps  ensure  that 
operations  are  threat-representative  and  that  reporting  and  data 
collection  are  to  standard.  USPACOM  and  USNORTHCOM 
have  established  Standing  Ground  Rules  that  allow  for  PCO 
activities  in  their  theaters. 

In  addition  to  ongoing  assessment  activities  in  FY15,  the  PCO 
supported  cybersecurity  operational  tests  of  acquisition  programs, 
and  an  Office  of  Cost  Assessment  and  Program  Evaluation  study. 
The  PCO  provided  this  support  in  less  time  than  a  traditional 
Red  Team  could  have  due  to  the  continual  reconnaissance 
and  network  accesses  that  had  already  been  authorized  and 
established.  This  approach  reduced  the  workload  for  Red  Teams 
that  are  in  high  demand.  During  FY15,  the  PCO,  operating 
outside  of  a  formal  test  or  exercise  period,  also  identified  an 
important  vulnerability  in  networks  supporting  USPACOM. 

The  PCO  provided  network  authorities  the  technical  details  and 
operational  implications  of  the  vulnerability,  worked  with  those 
authorities  to  identify  solutions,  and  verified  the  vulnerability  had 
been  resolved  in  subsequent  observations. 

The  PCO  provides  frequent  and  detailed  reporting  on  PCO 
operations  and  identified  vulnerabilities,  and  works  with  network 
authorities  and  CPTs  to  identify  and  implement  solutions  or 
mitigations.  The  PCO  will  verify  the  solutions  or  mitigations 
have  been  effectively  implemented  during  follow-on  operations. 
DOT&E  has  urged  the  leadership  at  other  CCMDs  to  establish 
Standing  Ground  Rules  to  enable  PCO  operations  in  their 
theaters. 

Advanced  Cyber  OPFOR  (ACO) 

The  tool  and  skill  sets  of  the  Cyber  Red  Teams  are  not  keeping 
pace  with  state-of-the-art,  nation-state  threats,  and  their 
operations  tempo  provides  little  time  for  operators  to  gain 
expertise  with  new  tools  or  to  learn  exploits  against  non-standard 
systems.  Furthermore,  it  is  difficult  for  them  to  obtain  advanced 
cyber  tools  through  normal  procurement  processes. 

DOT&E  created  the  Advanced  Cyber  OPFOR  (ACO)  concept 
to  augment  DOD  Red  Teams  and  the  PCO  with  advanced 
capabilities  our  cyber  adversaries  likely  possess.  The  ACO 
enables  developers  of  advanced  cyber  capabilities  and 
practitioners  of  advanced  techniques  to  assist  in  planning  and 
execution  of  PCO  operations.  For  example,  during  one  FY15 
exercise,  the  ACO  provided  capabilities  from  two  developers 
to  enable  the  PCO  to  traverse  defensive  infrastructure,  which 
had  been  impeding  PCO  network  attacks.  The  ACO  assist  was 
warranted  because  network  defenses  improved,  previous  exploits 
from  the  public  domain  no  longer  worked,  and  the  intelligence 


community  assessed  the  representative  adversary  to  possess  more 
advanced  tools  and  techniques.  DOT&E  will  employ  the  ACO 
routinely  during  FY16  in  support  of  the  PCO  in  similar  situations. 

Cyber  Threat  Assessments 

DOT&E  remains  engaged  with  key  intelligence  agencies  to 
ensure  the  latest  cyber  intelligence  is  incorporated  into  the 
planning  for  operational  tests  and  cybersecurity  assessments.  The 
Defense  Intelligence  Agency’s  Exercise  Support  Team  provides 
cyber  adversary  threat  assessments,  writes  realistic  cyber 
scenarios  to  support  CCMD  exercises,  and  provides  the  cyber 
threat  lead  during  these  exercises. 

As  network  defenses  continue  to  improve,  the  Intelligence 
Community  will  need  to  credit  advanced  cyber  adversaries 
with  capabilities  that  have  not  been  observed  in  employment, 
but  which  are  known  to  exist.  Additionally,  the  Intelligence 
Community  will  need  to  improve  the  characterization  of 
adversary  cyber  actions,  which  are  expected  during  wartime.  An 
adversary  may  reasonably  limit  cyber  activities  to  development 
of  accesses  and  exfiltration  of  information  during  peacetime,  but 
more  aggressive  cyber  activities  may  be  expected  when  major 
combat  platforms  are  committed  and  force-on-force  operations 
are  underway. 

Testing  of  Industrial  Control  Systems 

DOT&E  is  preparing  to  assess  acquisition  programs  that  employ 
commercial  industrial  control  systems.  DOT&E  commissioned 
testing  of  four  common  industrial  control  systems  with  the 
help  of  Sandia  National  Labs,  Pacific  Northwest  National 
Labs,  and  the  Johns  Hopkins  University  Applied  Physics  Lab. 
When  complete,  DOT&E  will  use  the  results  from  these  tests 
to  recommend  test  procedures,  and  will  provide  the  results  to 
programs  to  support  development  of  mitigation  strategies  for 
discovered  cyber  vulnerabilities.  DOT&E  will  also  make  the  test 
environments  and  virtual  instantiations  available  to  the  Cyber 
Mission  Force  and  supporting  cyber  ranges. 

DOD  Cyber  Strategy 

DOT&E  is  participating  in  three  Lines  of  Effort  from  the  DOD 
Cyber  Strategy: 

•  Exercise  Assessments.  In  coordination  with  the  Joint  Staff, 
DOT&E  will  assess  the  ability  of  CCMDs  to  sustain  critical 
missions  in  a  cyber-contested  environment.  Activities  led  by 
DOT&E  such  as  the  PCO  and  CAMP  development  (discussed 
above)  will  assist  in  providing  assessment  opportunities  and 
results. 

•  Computer  Network  Defense  Metrics  and  Evaluation.  Most 
systems  rely  on  the  host  network  or  environment  for 
cybersecurity  protection.  Additionally,  in  most  cases,  the 
Cybersecurity  Defense  Service  Providers  (CDSP)  assumes 
the  majority  of  key  cyber  defensive  responsibilities  and 
tasks.  Therefore,  measuring  the  effectiveness  of  CDSP 
capabilities  is  essential  to  evaluating  the  cybersecurity  posture 
of  every  system.  To  that  end,  DOT&E  will  participate  in  the 
development  of  metrics  and  test  methods  to  measure  CDSP 
performance. 


Cybersecurity  393 


F Y  1  5  CYBERSECURITY 


•  Red  Team  Oversight.  DOT&E,  in  coordination  with  the 
Chairman,  Joint  Chiefs  of  Staff,  will  establish  an  oversight 
system  of  all  DOD  Cyber  Red  Teams;  these  include  opposing 
force  aggressors,  cyber  system  penetration  testers,  and  teams 
that  perform  operational  cybersecurity  assessments.  The 
demand  for  Cyber  Red  Teams  in  all  three  of  these  primary 
roles  has  grown  over  the  past  several  years,  and  DOT&E 
projects  the  demand  will  continue  to  grow.  This  effort  will 
help  ensure  that  cyber  Red  Teams  are  resourced,  organized, 
trained,  and  equipped  to  effectively  meet  the  increasing 
mission  requirements. 

Cyber  Protection  Team  (CPT)  Training  and  Experimentation 

DOT&E,  in  partnership  with  USCYBERCOM  J7,  conducted  a 
pilot  Collective  Team  Training  course  for  CPTs  from  April  13 
through  May  19,  2015,  at  Camp  Dawson,  West  Virginia. 
Participants  included  three,  15-man  Quick  Reaction  Force  (QRF) 
cyber  teams  from  both  Army  and  Navy  CPT  units.  Students 
were  initially  trained  in  four  functional  cyber  defensive  groups 
(Harden,  Monitor,  Coordinate,  and  Pursue)  and  then  brought 
together  into  a  QRF  team  construct.  DOT&E  evaluated  the 
performance  of  the  three  QRF  teams  during  force-on-force 
cyber  engagements  designed  to  simulate  typical  CPT  mission 
deployments. 

DOT&E  conducted  a  follow-up  CPT  Performance  Assessment 
Experiment  in  July  and  August  2015  to  compare  performance  of 
CPTs  that  received  collective  team  training  with  CPTs  that  had 
no  team  training.  Emerging  results  indicate  CPTs  that  received 
team  training  performed  significantly  better  than  those  without 
training.  DOT&E  also  observed  that  some  individuals  assigned 
to  CPTs  do  not  possess  the  proper  training,  background,  or 
motivation  to  become  effective  CPT  members.  DOT&E  acquired 
and  enhanced  survey  tools  that  can  help  determine  individual 
suitability  for  CPTs,  and  offered  these  tools  to  USCYBERCOM 
and  Service  cyber  components. 

DOT&E  will  provide  the  aggregated  results  to  USCYBERCOM 
and  the  Service  cyber  components  to  help  inform  decisions 
regarding  future  CPT  training.  DOT&E  expects  that  significant 
time  on  realistic  ranges  will  be  instrumental  to  CPTs  attaining  an 
effective  operational  capability.  This  effort  also  demonstrated 
that  an  unclassified  range  has  cost,  schedule,  and  availability 
advantages  over  a  classified  range  for  a  subset  of  CPT  training 
needs. 

Cyber  Ranges 

The  DOD  Enterprise  Cyber  Range  Environment  is  a  collection 
of  four  independent  cyber  range  assets  where  classified  training 
and  testing  can  occur.  In  2011,  these  ranges  were  experiencing 
budget  cuts  and  were  becoming  unsustainable.  DOT&E 
proposed  critical  enhancements  for  these  cyber  ranges  and 
the  establishment  of  an  EA  in  2012;  additional  funding  was 
programmed  in  the  FY13  Program  Review,  but  there  was  no 
decision  for  an  EA. 

The  FY15  NDAA  directed  DOD  to  establish  an  EA  for  cyber 
training  ranges  and  an  EA  for  cyber  testing  ranges;  the  NDAA 
does  not  preclude  the  EAs  from  being  a  single  entity.  As 


combined  testing  and  training  are  mandatory  for  the  ranges’ 
efficient  use,  and  more  importantly  for  keeping  pace  with  the 
rapidly  evolving  cyber  threats,  the  creation  of  two  separate 
EAs — with  separate  responsibilities  and  incentives — would  make 
it  difficult  to  continue  to  conduct  combined  activities  in  a  timely 
manner.  Despite  this,  the  Department  appears  to  be  on  a  path  to 
create  two  separate  EAs.  This  will  likely  hinder  the  Department’s 
ability  to  respond  to  rapidly  evolving  and  increasingly 
sophisticated  cyber  threats.  In  order  to  provide  the  optimal  cyber 
range  posture  for  the  DOD,  a  single  EA  should  be  designated  for 
cyber  ranges  with  the  authority  to  oversee  funding  and  personnel 
for  all  DOD-owned  cyber  ranges,  and  the  authority  to  identify 
and  certify  commercial  cyber  range  resources  for  DOD  use,  as 
appropriate. 

Observations 

In  FY15,  cybersecurity  assessment  teams  consistently  identified 
vulnerabilities  which  place  DOD  missions  at  high  risk  from  cyber 
compromise,  exploitation,  and  disruption.  Although  mission 
impacts  are  not  always  permitted  during  exercises,  DOT&E 
assesses  the  likelihood  and  magnitude  of  impacts  to  missions 
based  on  accesses  achieved  by  the  Cyber  OPFOR.  In  many 
cases,  DOT&E  has  assessed  that  catastrophic  kinetic  impacts 
could  be  enabled  by  the  information  the  Cyber  OPFOR  has 
accessed. 

The  limitations  imposed  upon  the  Cyber  OPFOR  by  exercise 
authorities  continue  to  reduce  the  value  of  both  cybersecurity 
assessments  and  training  of  the  Service  member  and  network 
defenders.  Exercise  authorities  for  several  CCMDs  are  working 
with  DOT&E  and  assessment  teams  to  identify  or  develop  better 
venues  in  which  cyber  effects  can  be  demonstrated  to  stress 
networks,  defenses,  and  missions.  In  light  of  well-publicized 
intrusions  into  U.S.  Government  networks  this  fiscal  year, 
exercise  authorities  should  move  aggressively  to  maximize 
training  in  realistic  cyber-threat  environments. 

DOT&E  observed  a  continued  increase  in  the  participation 
of  CDSPs  during  FY15  exercises,  and  also  noted  growing 
involvement  by  CPTs.  Although  local  network  defenders, 

CDSPs,  and  CPTs  need  to  work  to  optimize  their  combined 
efforts,  DOT&E  has  observed  that  some  cyber  attacks  were 
less  effective  in  FY15  than  in  previous  years.  The  following 
paragraphs  discuss  protective  measures  and  reactive  capabilities 
that  were  observed  in  FY15. 

Protective  Cyber  Defense  -  Hindering  Attacks.  The  first  line 
of  cyber  defenses  involves  configuring  networks  and  systems 
to  prevent  or  hinder  access  by  unauthorized  parties.  In  FY15, 
assessment  teams  continued  to  find  problems  with  software 
configuration  and  outdated  patches,  but  also  confirmed  that 
networks  with  up-to-date  patches  and  configuration  best-practices 
noticeably  hindered  the  Cyber  OPFOR  from  gaining  network 
access.  Assessment  teams  also  reported  that  successful  attacks 
tended  to  exploit  common  information  infrastructure  via  stolen  or 
default  credentials,  and  services  such  as  email,  SharePoint,  and 
web  portals. 
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Compared  to  previous  years,  assessment  teams  observed  an 
increasing  number  of  events  where  protective  defenses  thwarted 
lower-capability  attacks.  Phishing  attacks  were  less  successful  in 
networks  where  email  links  to  internet  addresses  were  disabled; 
another  reason  for  improvements  is  likely  the  heightened 
awareness  and  focused  training  that  followed  the  publicized 
intrusions  on  U.S.  Government  networks. 

Network  defenders  and  CCMDs  should  continue  to  improve 
their  defensive  posture  and  recognize  that  more  advanced  threat 
capabilities  exist.  DOT&E  is  working  with  the  PCO  to  develop 
more  advanced  tools  and  techniques  that  are  representative  of 
advanced  adversary  capabilities,  which  will  begin  to  be  employed 
during  FY16  assessments. 

Reactive  Defense  -  Responding  to  Attacks.  Reactive  defense 
involves  the  detection  and  response  to  cyber  adversaries  that 
have  penetrated  the  protective  defenses  and  are  operating  within 
the  networks  and  systems.  Defenders  typically  rely  on  detecting 
the  signatures  of  known  exploits,  noticing  unusual  activities, 
or  responding  to  the  effects  of  an  attack  after  it  occurs.  In  one 
exercise  example,  defenders  identified  an  intrusion  and  conducted 
a  coordinated  response  to  shut  down  a  Cyber  OPFOR  access 
point.  The  actions  were  effective  against  the  original  access 
point,  but  not  sufficiently  timely  as  the  cyber  OPFOR  had  already 
maneuvered  to  another  foothold. 

Early  detections  are  critical  to  reduce  the  adversary’s  opportunity 
to  obtain  additional  network  privileges  and  move  to  additional 
network  footholds.  In  another  exercise,  the  assessment  team 
observed  effective  and  timely  collaboration  across  operators  and 
network  defenders:  an  operator  noticed  an  unusual  change  to  a 
situational-awareness  data  display,  reported  the  discovery  while 
correcting  the  errors,  and  rapidly  notified  the  network  defenders 
who  were  able  to  thwart  the  cyber  attack. 

DOT&E  engaged  extensively  with  CPTs  in  FY15,  developing 
a  better  understanding  of  their  mission,  how  they  will 
support  network  defense,  and  metrics  for  assessment  of  their 
performance.  DOT&E  also  began  assessment  of  CCMD 
processes  associated  with  supporting  offensive  cyber  operations. 

Key  Findings 

CCMDs  need  an  integrated  and  reactive  cyber  defense  that 
supplements  proper  configurations,  up-to-date  software,  and 
signature-based  tools.  In  order  to  be  effective  against  an 
advanced  persistent  threat,  CCMDs  will  need  to  be  supported  by: 

•  Improved  detection  of  non-signature-based  activities  such  as 
exfiltration  and  unauthorized  authenticated  access 

•  Accurate  cyber  situational  awareness  and  timely  reporting  to 
enable  correlation  of  information  to  identify  trends  and  attacks 

•  Effective  response  capabilities  and  playbooks  to  quickly 
upgrade  defenses  via  local  defenders,  CDSPs,  or  CPTs 


•  Timely  response  actions  by  counter-cyber  elements  of  the 
Cyber  Mission  Force 

In  the  course  of  both  operational  tests  and  exercise  or  site 
assessments,  the  assessing  organizations  often  identify 
vulnerabilities,  practices  (good  and  bad),  and  tools  that  may  have 
enterprise  implications  and  merit  senior  leadership  review  and 
action.  For  these  vulnerabilities  and  enhancements,  DOT&E 
publishes  finding  memoranda  to  the  DOD  entities  best  able  to 
address  the  situation.  In  FY15,  DOT&E  initiated  or  published 
research  on  the  following  topics: 

•  Host  Based  Security  System  -  DOT&E  identified  newly  found 
shortfalls  in  the  use  of  this  enterprise-wide  tool.  The  Defense 
Information  Systems  Agency  (DISA)  has  acknowledged  and 
addressed  these  findings. 

•  Special  Handling  Documents  -  DOT&E  identified  shortfalls  in 
the  procedures  for  electronically  transmitting  special-handling 
documents.  The  Joint  Staff  and  Undersecretary  of  Defense  for 
Policy  have  acknowledged  and  addressed  these  findings. 

•  Shipboard  Tactical  Systems  -  DOT&E  identified 
vulnerabilities  with  the  tactical  datalinks  supporting  afloat 
platforms.  The  Navy  has  acknowledged  and  addressed  these 
findings. 

•  Java  Beans  Open  Systems  Software  -  DOT&E  identified  a 
number  of  common  vulnerabilities  in  this  widely  used  software 
platform.  The  DOD  CIO  and  DISA  are  reviewing  solutions. 

•  Industrial  Control  Systems  -  DOT&E  identified  common 
vulnerabilities  in  key  components  of  these  systems.  The  Joint 
Staff,  DOD  CIO,  DISA,  and  the  Services  are  investigating 
solutions  to  the  issues  identified.  DOT&E  is  sharing  test 
data  with  the  Office  of  Naval  Research  on  new  technology 
developments  related  to  protecting  key  control  system 
components. 

•  Information  Condition  (INFOCON)  Guidance  -  DOT&E 
is  researching  contradictory  or  incomplete  guidance  for 
implementing  INFOCON  changes  that  reflect  heightened 
cybersecurity  states  based  on  detected  or  anticipated 
cyber-adversary  actions. 

•  Cyber  tools  -  DOT&E  is  researching  problems  found  with  the 
use  of  Kerberos  authentication,  the  availability  of  PowerShell 
utilities,  and  the  effectiveness  of  software  whitelisting  and 
management  tools  such  as  AppLocker  and  Bit9. 

Future  Assessments 

DOT&E  plans  to  focus  its  FY16  assessment  resources  on  those 
CCMDs  who  are  willing  to  permit  realistic  cyber  effects  during 
major  exercises  and  commit  to  the  development  of  CAMPs 
and  TCRCs.  Table  2  provides  a  list  of  currently  planned  FY16 
assessments. 
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Test  and  Evaluation  Resources 


Public  law  requires  DOT&E  to  assess  the  adequacy  of 
operational  and  live  fire  testing  conducted  for  programs  under 
oversight,  and  to  include  comments  and  recommendations 
on  resources  and  facilities  available  for  OT&E  and  on  levels 
of  funding  made  available  for  OT&E  activities.  DOT&E 
monitors  and  reviews  DOD  and  Service-level  strategic  plans, 
investment  programs,  and  resource  management  decisions  to 
ensure  capabilities  necessary  for  realistic  operational  tests  are 
supported.  This  report  highlights  general  areas  of  concern  in 
testing  current  systems  and  discusses  significant  issues,  DOT&E 
recommendations,  and  T&E  resource  and  infrastructure  needs 
to  support  operational  and  live  fire  testing.  FY15  focus  areas 
include: 

•  Army  Support  of  OT&E 

•  Cybersecurity  Red  Team  Personnel  Shortfalls 

•  Cyber  Threat  Support  to  T&E 

•  High  Altitude  Electromagnetic  Pulse  (HEMP)  Test  Capability 

•  Joint  Strike  Fighter  (JSF)  Advanced  Electronic  Warfare  (EW) 
Test  Resources 

•  Point  Mugu  Sea  Test  Range  (STR)  Enhancements  to  Support 
OT&E  of  Air  Warfare  Programs 

•  EW  for  Land  Combat 

•  Navy  Advanced  EW  Test  Resources  and  Environments 


•  Equipping  Self-Defense  Test  Ship  (SDTS)  for  Aegis  Combat 
System,  Air  and  Missile  Defense  Radar  (AMDR)  and  Evolved 
SeaSparrow  Missile  (ESSM)  Block  2  Operational  Testing 

•  Multi-Stage  Supersonic  Targets  (MSST) 

•  F ifth-Generation  Aerial  Target 

•  Warrior  Injury  Assessment  Manikin  (WIAMan) 

•  Torpedo  Surrogates  for  Operational  Testing  of  Anti-Submarine 
Warfare  (ASW)  Platforms  and  Systems 

•  Submarine  Surrogates  for  Operational  Testing  of  Lightweight 
and  Heavyweight  Torpedoes 

•  Signature  Data  Collection  for  Infrared  (IR)  Guided  Surface  to 
Air  and  Hostile  Fire  Threats  to  Support  Model  Development 

•  Threat  Modeling  and  Simulation  (M&S)  to  Support  Aircraft 
Survivability  Equipment  (ASE)  Testing 

•  Foreign  Materiel  Acquisition  Support  for  T&E 

•  Tactical  Engagement  Simulation  with  Real  Time  Casualty 
Assessment  (TES/RTCA) 

•  Testing  in  Urban  Environments 

•  Biological  Defense  Testing  at  West  Desert  Test  Center  on 
Dugway  Proving  Ground,  Utah 

•  Range  Sustainability 

•  Continuing  Radio  Frequency  Spectrum  Concerns 


Army  Support  of  OT&E 

In  the  2014  Annual  Report,  DOT&E  recommended  that  the 
Army  restore  the  Operational  Test  Command  (OTC)  and  Army 
Evaluation  Center  (AEC)  budgets  in  order  to  maintain  FY14 
staffing  levels,  and  ensure  staffing  levels  of  the  Army  T&E 
Executive  are  consistent  with  its  mission.  In  a  memorandum 
to  the  Secretary  of  the  Army,  dated  November  12,  2014, 

DOT&E  highlighted  the  importance  of  the  office  of  the  Army 
T&E  Executive  and  recommended  decisions  to  reduce  staff 
be  reversed.  For  FY16,  the  OTC  budget  has  remained  flat  and 
the  AEC  budget  has  been  reduced  an  additional  4  percent  from 
FY15.  Staffing  levels  at  OTC  have  increased  ~7  percent,  and 
AEC  staffing  has  decreased  ~1  percent  compared  to  FY14  levels. 

During  the  DOT&E  review  of  the  Army’s  T&E  budget  and 
resources,  the  Army  acknowledged  that  the  current  staffing 
levels  may  cause  increased  customer  billing  rates,  the  inability 
to  conduct  simultaneous  operational  test  events,  and  longer 
timelines  for  the  release  of  test  reports.  DOT&E  is  concerned 
that  the  reduced  staffing  equates  to  an  inadequate  number  of 
experienced  T&E  staff  needed  to  ensure  efficient  and  timely 
preparation  of  TEMPs  and  test  plans.  Delays  in  test  planning, 
execution,  and  reporting  can  result  in  delayed  acquisition.  The 
savings  generated  by  further  reducing  the  staff  of  OTC  and  AEC 
could  be  offset  by  cost  penalties  to  acquisition  programs  that 


are  proportional  to  their  respective  spend  rates  multiplied  by 
the  duration  of  delay.  Other  smaller  but  valuable  programs  may 
be  delayed  even  longer,  as  priority  will  be  given  to  the  Major 
Defense  Acquisition  Programs. 

DOT&E  will  continue  to  monitor  the  Army  T&E  workforce  to 
ensure  that  it  is  able  to  support  and  not  hinder  the  outcomes  of 
the  Army’s  acquisition  programs. 

Cybersecurity  Red  Team  Personnel  Shortfalls 

The  increasing  demand  for  certified  cyber  red  teams  to 
support  training,  operations,  and  acquisition  testing  is  placing 
considerable  strain  on  this  small  professional  community 
within  the  Department.  This  demand  is  driven  by  the 
growing  desire  for  acquisition  program  managers  to  test 
their  systems  during  development  to  discover  and  address 
cybersecurity  vulnerabilities,  the  continuing  need  to  perform 
threat-representative  cybersecurity  operational  testing,  and  the 
cybersecurity  training  needs  of  the  growing  number  of  Cyber 
Mission  Force  personnel.  The  subset  of  red  team  personnel 
certified  to  operate  on  live  networks  are  critical  to  conducting 
the  operational  testing  and  Combatant  Command  (CCMD) 
assessments  described  in  the  Cybersecurity  section  of  this  report. 
These  highly  qualified  red  teams  are  experiencing  the  highest 
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demand,  and  some  of  these  teams  have  indicated  to  DOT&E 
that  they  will  not  be  able  to  support  all  of  the  currently  planned 
operational  tests. 

DOT&E  has  already  seen  instances  in  which  tests  were 
rescheduled  or  could  not  be  performed  as  planned  due  to  a  lack  of 
available  cyber  teams  authorized  to  conduct  cyber  operations  on 
live  networks  and  enclaves.  The  high  operational  tempo  of  the 
red  teams  has  reduced  or  eliminated  opportunities  for  the  teams 
to  train,  thereby  eroding  their  ability  to  ensure  their  skill  level  is 
commensurate  with  advanced  nation  state  cyber  threats.  The  high 
operational  tempo  has  also  induced  a  number  of  experienced  red 
team  members  to  seek  higher  paying,  lower  demand  jobs  outside 
of  the  Department,  further  exacerbating  the  personnel  shortfalls. 

A  number  of  initiatives  would  help  address  the  increasing 
shortfall  of  cyber  red  team  personnel: 

•  Creating  pay  and  other  incentives  for  cybersecurity  personnel 
such  as  those  afforded  other  highly-trained,  critical  DOD 
personnel  (e.g.  pilots) 

•  Creating  a  “Persistent  Cyber  Opposing  Force  (OPFOR)” 
composed  of  red  team  members  from  across  the  Services  to 
provide  efficient,  flexible,  and  threat-realistic  cyber  effects 

•  Establishment  of  dedicated  T&E  cyber  teams,  core-funded, 
rather  than  program-funded,  to  preserve  continuity  of  skills 

•  Creating  and  implementing  “red  team  in  a  box”  software 
which  can  automatically  identify  common  cybersecurity 
vulnerabilities 

Cyber  Threat  Support  to  T&E 

DOT&E  has  recognized  that  the  cyber  threat  has  expanded  into 
the  wireless  domain  using  radio  frequency  (RF)  transmissions  to 
deliver  the  threat  effect  to  the  intended  victim.  This  new  medium 
of  delivery  has  expanded  the  testing  required  to  define,  analyze, 
and  resolve  U.S.  weapon  system  vulnerabilities.  The  expansion 
of  cyber  threat  delivery  into  the  RF  domain  has  created  a  much 
more  diverse  EW  portfolio  and  has  led  the  defense  industry  to 
begin  recognizing  the  merging  of  cyber  and  EW  into  a  much 
more  diverse  threat. 

The  $5.0  Million  appropriations  increase  for  Threat  Resource 
Activities  allowed  DOT&E  to  expand  its  understanding  of 
the  “wireless”  cyber  threat  and  begin  the  process  of  defining, 
cataloging,  and  incorporating  these  threats  into  a  classified, 
online  Threat  Database  available  to  the  Department  in  support 
of  U.S.  weapon  system  testing.  This  online  tool  defines  the 
threat,  provides  the  appropriate  contact  information  for  the 
responsible  intelligence  organization,  and  the  status  of  available 
representations  of  that  threat  to  include  models  and  simulations, 
surrogates,  and/or  hardware/software  representations. 

DOT&E  recognized  that  efficiencies  in  our  operations  could 
be  gained  by  merging  this  cyber  threat  activity  with  ongoing 
cybersecurity  activities  supporting  the  CCMD  and  Service 
assessments  as  part  of  the  Cybersecurity  Assessment  Program. 
DOT&E’s  Cyber  Threat  Folder  developers,  located  within 
Defense  Intelligence  Agency  (DIA),  were  transferred  to  the 
Threat  Resource  Activity  to  allow  for  continued  residence  in  DIA 
Fleadquarters  and  direct  access  to  all  pertinent  cyber  threat  data. 


These  cyber  threat  analysts  continue  to  support  the  CCMD 
Cybersecurity  Assessment  Program  while  also  providing  threat 
data  for  incorporation  into  DOT&E’s  online  Threat  Database. 
While  working  with  the  DOT&E  analysts  at  DIA  Headquarters, 
the  Threat  Resource  Activity  recognized  there  was  a  pressing 
need  for  the  intelligence  community  and  the  users  of  cyber  threat 
information  to  have  a  process  for  easily  sharing  Cyber  Threat 
Folders.  The  Threat  Resource  Activity  began  the  process  of 
developing  a  Cyber  Threat  Folder  repository  that  would  allow 
intelligence  organizations  and  the  testing  community  to  have 
access  to  Service  and  intelligence  organization  Cyber  Threat 
Information.  This  is  an  ongoing  activity  that  will  be  “Beta 
Tested”  inFY16. 

High  Altitude  Electromagnetic  Pulse  (HEMP)  Test  Capability 

DDG  51  Ship  Specification,  Section  407  establishes  requirements 
for  DDG  5 1  Electromagnetic  Pulse  (EMP)  Protection.  Section 
407  states  that  during  the  guarantee  period  of  the  ship,  the 
Government  will  conduct  a  full  ship  EMP  test  to  determine  the 
performance  of  the  ship’s  electronic  systems  under  simulated 
EMP  conditions. 

The  Navy  currently  does  not  have  a  capability  to  conduct  a 
survivability  assessment  of  a  full  ship  to  EMP  effects.  Current 
Navy  practice  is  to  conduct  limited  testing  on  ship  systems  and 
sub-systems,  and  then  extrapolate  these  results  to  the  entire  ship. 
This  test  approach  is  not  technically  effective  nor  cost  efficient 
since  it  is  limited  in  scope,  time  consuming,  and  expensive  due 
to  the  time  required  to  complete  testing  a  handful  of  spaces. 

More  importantly,  this  testing  methodology  is  not  performed  at 
sea  in  an  operational  mode  and  doesn’t  provide  the  data  needed 
to  adequately  assess  the  full  ship  EMP  survivability.  Existing 
EMP  modeling  and  simulation  capabilities  provide  very  limited 
information  on  ship  survivability  with  significant  uncertainties. 

After  a  detailed  assessment  of  current  OSD  nuclear  range 
capabilities,  the  OSD  Chemical  Biological  Radiological 
and  Nuclear  Survivability  Oversight  Group  -  Nuclear 
(CSOG-N)  T&E  Working  Group  Roadmap  identified  a  full 
ship  EMP  Threat-Level  Simulator  (TLS)  for  warships  as 
their  most  important  T&E  gap.  Additionally,  the  Tri-Service 
Technical  Working  Group  responsible  for  the  development  of 
MIL-STD-4023,  HEMP  Protection  for  Military  Surface  Ships, 
agreed  that  a  full  ship  EMP  TLS  is  required  for  warship  EMP 
threat  survivability  assurance. 

The  Chief  of  Naval  Research/Director,  Innovation,  Technology 
Requirements,  and  T&E  (N84)  has  teamed  with  the  Defense 
Threat  Reduction  Agency  to  establish  a  Ship  EMP  TLS  Test 
Working  Group  to  inform  Navy  leadership  of  the  increasing 
criticality  of  this  threat.  The  Defense  Threat  Reduction 
Agency  determined  that  tests  using  a  full  ship  EMP  TLS  is 
the  best  approach  to  demonstrate  ship  threat-level  HEMP 
protection  and  mission  assurance  in  accordance  with  standing 
Navy  requirements.  The  costs  to  build  a  full  ship  EMP 
TLS  capability  are  estimated  to  be  $49  -  54  Million.  Once 
operational,  the  costs  to  conduct  the  first  nine  tests  are  estimated 
at  $17.5  -  $18.6  Million.  Full  ship  EMP  TLS  testing  at  sea 
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will  support  mission  assurance  by  provide  test  data  for  HEMP 
modeling  and  realistic  HEMP  training  scenarios  for  ship  crews. 
At  sea  testing  using  this  capability  will  demonstrate  full  ship 
EMP  survivability  and  support  the  U.S.  nuclear  deterrent  posture. 

Joint  Strike  Fighter  (JSF)  Advanced  EWTest  Resources 

Since  February  2012,  when  DOT&E  identified  shortfalls  in 
EW  test  resources,  significant  progress  has  been  made  in  some 
instances,  while  progress  is  lacking  in  other  areas.  The  EW 
assets  being  purchased  are  key  to  the  development,  testing,  and 
timely  fielding  of  numerous  U.S.  systems  critical  to  operating 


successfully  against  threats  that  currently  exist,  are  proliferating, 
or  are  undergoing  an  accelerating  pace  of  significant  upgrades. 
These  systems  include  the  JSF,  F-22  Increment  3.2  A/B,  B-2 
Defensive  Management  System,  Long-Range  Strike  Bomber,  and 
the  Next  Generation  Jammer  for  the  EA-18G.  The  status  of  these 
EW  upgrades  is  displayed  in  the  Table  immediately  below. 

Due  to  delays  and  inaction  by  the  F-35  Joint  Program  Office, 
the  situation  at  the  JSF  mission  data  file  reprogramming  lab  has 
reached  a  critical,  nearly  unrecoverable  point. 


TABLE  1.  RECOMMENDATIONS  ON  ELECTRONIC  WARFARE  TEST  RESOURCES 

DOT&E  Recommendation 

Current  Status 

Developing  a  combination  of  open- and  closed-loop  simulators  in  the 
numbers  required  for  operationally  realistic  open-air  range  testing  of  JSF  and 
other  systems  beginning  in  2018. 

Both  the  open-  and  closed-loop  efforts  are  underway. 

The  open-loop  effort  delivers  nine  systems  between  mid-2016  and  mid-201 7; 
and  is  planned  to  provide  an  additional  7,  for  a  total  of  1 6,  in  time  to  support 

F-35  IOT&E  and  other  testing  in  201 8  and  beyond.  Delivery  of  the  first  two 
open-loop  systems  is  expected  by  mid-201 6. 

The  closed-loop  effort  is  also  underway,  but  the  mobile  closed-loop  systems 
required  for  operational  testing  are  not  scheduled  to  be  available  until  mid-  to 
late-2019,  well  after  the  planned  F-35  IOT&E.  The  architecture  of  the  open-loop 
systems  will  provide  adequate  test  capabilities  for  F-35  Block  3F  IOT&E,  in  lieu  of 
closed-loop  systems. 

Upgrading  the  government  anechoic  chambers  with  adequate  numbers  of 
signal  generators  for  realistic  threat  density. 

Initial  studies  of  materiel  solutions  to  achieve  realistic  densities  have  begun: 

•  The  Navy  chamber  has  procured  initial  test  support  equipment  for  direct 
injection  capability  and  executed  a  limited  F-35  EW  test  in  September  2014. 

•  The  Air  Force  chamber  has  identified  a  path  forward  covering  extensive 
upgrades  through  2020. 

•  The  JSF  program  has  yet  to  develop  concrete  plans  to  integrate  chamber 
testing  into  the  verification  test  strategy. 

Upgrading  the  JSF  mission  data  file  reprogramming  lab  to  include  realistic 
threats  in  realistic  numbers. 

A  JSF  Program  Office-sponsored  study  to  determine  upgrade  requirements  was 
completed  in  December  2014.  It  confirmed  the  shortfalls  identified  by  DOT&E 
in  February  201 2,  but  also  identified  many  other  critical  shortfalls  preventing 
effective  and  efficient  mission  data  file  development  and  reprogramming. 
Unfortunately,  inexplicable  delays  by  the  program  since  this  study  was 
completed  have  resulted  in  little  to  no  progress  in  addressing  these  shortfalls. 
Also,  the  program  plans  to  procure  fewer  signal  generators  than  the  study 
recommended,  further  jeopardizing  the  program's  ability  to  generate  effective 
mission  data  for  IOT&E  and  Block  3F  operations. 

Providing  Integrated  Technical  Evaluation  and  Analysis  of  Multiple  Sources 
intelligence  products  needed  to  guide  threat  simulations. 

Products  have  been  completed  and  delivered,  and  are  being  used  to  support 
development  of  the  open-  and  closed-loop  threat  radar  simulators. 

Point  Mugu  Sea  Test  Range  (STR)  Enhancements  to  Support 
OT&E  of  Air  Warfare  Programs 

In  2015,  the  JSF  Joint  Operational  Test  Team  (JOTT)  determined 
that  an  ability  to  conduct  operational  test  missions  on  the  Point 
Mugu  STR  could  considerably  shorten  the  duration  of  F-35 
IOT&E,  the  pace  of  which  is  currently  constrained  by  the 
competition  with  other  programs  for  a  limited  number  of  range 
periods  available  each  week  at  the  Air  Force  Western  Test  Range 
(WTR),  in  Nevada.  Nearly  all  mission-level  testing  in  IOT&E 
was  originally  scheduled  to  take  place  at  the  WTR. 

The  JOTT  assessment  concluded  that  the  key  to  conducting 
F-35  IOT&E  missions  at  STR  was  the  timely  completion 
and  integration  of  the  Air  Warfare  Battle  Shaping  (AWBS) 
system  at  the  STR.  AWBS  is  a  variant  of  the  Air-to-Air  Range 
Instrumentation  system  at  the  WTR,  where  it  is  essential  for 


scoring  and  post-mission  reconstruction  and  analysis  of  OT&E 
missions.  At  the  time  of  the  assessment,  the  development  and 
integration  of  AWBS  at  the  STR  was  stalled  due  to  a  severe 
funding  shortfall.  In  response  to  the  JOTT  assessment,  DOT&E 
and  USD(AT&L)  together  allocated  $20  Million  to  fund  the 
shortfall. 

About  the  same  time  of  the  JOTT  assessment  and  the  DOT&E 
and  USD(AT&L)  decision  to  provide  funding  for  AWBS  at 
the  STR,  the  JSF  Program  Office  decided  to  discontinue  the 
Lockheed  Martin  Verification  Simulation  (VSim),  a  high-fidelity 
manned  simulation  central  to  the  program’s  operational  test  plans, 
and  transfer  responsibility  for  the  program’s  manned  simulator 
requirements  to  Naval  Air  Systems  Command  (NAVA1R).  The 
JSF  Program  Office  stopped  work  on  the  Lockheed  Martin  effort 
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due  to  severe  cost  overruns  and  their  assessment  that  Lockheed 
Martin  would  be  unable  to  deliver  an  adequate  VSim  capability 
in  time  for  F-35  IOT&E. 

DOT&E  is  convinced  that  NAVA1R  will  likewise  be  unable  to 
deliver  an  adequate  VSim  capability  in  time  for  F-35  IOT&E,  and 
that,  in  particular,  NAVAIR  will  be  unable  to  complete  the  project 
within  the  cost  constraints  imposed  by  the  Program  Office.  At 
the  same  time,  DOT&E  recognized  that  additional  infrastructure 
upgrades  for  the  Point  Mugu  STR,  in  addition  to  the  completion 
and  integration  of  AWBS,  would  be  required  to  make  the  STR  a 
robust  venue  of  F-35  operational  testing.  Specifically,  DOT&E 
determined  that  the  STR  needed  equipment  and  software  for 
replicating  the  air  surveillance  and  command  and  control 
infrastructures  of  a  threat  integrated  air  defense  system. 

Accordingly,  DOT&E  has  recommended  to  the  Secretary 
of  Defense  that  a  significant  portion  of  the  money  currently 
allocated  for  VSim  be  reallocated  to  constructing  the  required 
integrated  air  defense  system  infrastructure  at  the  STR.  DOT&E 
recommendations  include  buying  a  variety  of  systems,  a  number 
of  which  are  available  off-the-shelf  on  the  international  defense 
market. 

EW  for  Land  Combat 

Networked  mission  command  systems  that  support  the 
commander’s  mission  execution  across  the  Brigade  Combat 
Team  (BCT)  are  a  cornerstone  of  the  Army’s  modernization 
plan.  These  integrated  network  capabilities  are  distributed 
throughout  a  combat  formation  and  its  support  elements,  from 
the  brigade  command  posts  down  to  the  individual  dismounted 
Soldier.  Commanders  using  tactical  network  systems  have  the 
unprecedented  ability  to  transfer  information  such  as  voice, 
video,  text,  position  location  information,  and  high-resolution 
photographs  throughout  the  BCT,  and  provide  individual 
commanders  access  to  information  needed  to  complete  their 
mission.  The  expanded  use  of  radio  frequency  spectrum  to 
support  mission  command  systems  with  supporting  data  networks 
exposes  the  BCT  to  contemporary  EW  threat  vectors  available 
to  a  broad  range  of  potential  enemies.  As  the  Army  becomes 
more  dependent  on  these  sophisticated  network  technologies,  it 
is  critical  that  the  developmentaFoperational  test  communities 
continue  to  identify  and  assess  vulnerabilities  of  these  systems. 
Decision  makers  must  understand  the  inherent  vulnerabilities,  as 
well  as  the  ways  in  which  an  enemy  may  choose  to  exploit  and/or 
degrade  the  network. 

During  operational  testing,  threat  EW  is  part  of  a  broader 
capability  set  that  is  made  available  to  the  OPFOR  commander. 
Ideally,  the  EW  capabilities,  tactics,  techniques,  and  procedures 
employed  by  the  OPFOR  during  test  should  represent  those 
of  our  potential  adversaries.  At  present,  there  are  necessary 
and  severe  limitations  placed  on  the  location,  frequency,  time, 
and  amount  of  power  that  may  be  emitted  by  the  threat  EW 
equipment,  in  order  to  avoid  interference  with  commercial 
aircraft  and  the  civilian  populations  adjacent  to  the  test  and 
training  ranges.  Realistic  threat  EW  against  communication 
satellites  is  not  allowed  during  operational  testing  due  to  the 


potential  of  interfering  with  satellites  supporting  commercial  and 
military  operations.  These  limitations  cause  artificialities  in  the 
test  environment  and  affect  the  OPFOR’s  ability  to  degrade  the 
network  and  combine  EW  with  other  lethal  attacks.  DOT&E 
recommends  that  the  Army  continue  to  investigate  potential 
technical  and  procedural  solutions  to  the  current  limitations. 

These  critical  threat  test  capabilities  are  needed  to  support  testing 
of  Warfighter  Information  Network  -  Tactical  Increment  2, 

Nett  Warrior/Rifleman  Radio,  Mid-Tier  Networking  Vehicular 
Radio,  Manpack  Radio,  and  Joint  Battle  Command  -  Platform. 

Navy  Advanced  EWTest  Resources  and  Environments 

Capability  for  Realistic  Representation  of  Multiple  Anti-Ship  Cruise 
Missile  (ASCM)  Seekers  for  Surface  EW  Improvement  Program 
(SEWIP)  Operational  Testing 

This  gap  in  test  capability  was  initially  identified  in  DOT&E ’s 
FY13  Annual  Report  as  “Additional  Electronic  Warfare 
Simulator  Units  for  Surface  Electronic  Warfare  Improvement 
Program  (SEWIP)  Operational  Testing.”  The  Navy  addressed  it 
with  development  of  a  programmable  seeker  simulator  that  could 
represent  different  ASCM  seekers  by  specifying  the  electronic 
waveform  emission  characteristics  for  one  of  several  possible 
threats.  However,  the  effective  radiated  power  (ERP)  was  not 
among  those  characteristics,  resulting  in  simulated  attacks  by 
ASCM  representations  displaying  disparate  levels  of  ERP  that 
are  unlikely  to  be  encountered  during  a  stream  raid  attack  of  two 
ASCMs  (along  the  same  bearing  and  elevation  and  within  close 
proximity  of  one  another).  The  programmable  seeker  simulator, 
termed  the  “Complex  Arbitrary  Waveform  Synthesizer,”  needs 
to  be  modified  such  that  its  ERP  more  realistically  represents  the 
second  ASCM  of  a  dual  ASCM  stream  raid. 

The  next  SEWIP  Block  2  OT&E  is  projected  for  FY19.  This  is  to 
be  followed  by  FOT&E  on  a  Product  Line  Architecture  compliant 
DDG  51  with  Block  2  actually  integrated  with  the  Aegis  Combat 
System.  This  integration  was  not  part  of  the  Block  2  IOT&E. 
Subsequent  FOT&E  would  be  with  the  DDG  1000  and  CVN  78 
combat  systems.  Estimated  cost  to  add  the  ERP  improvement  is 
$5.0  Million. 

Long-Term  Improvement  in  Fidelity  of  ASCM  Seeker/Autopilot 
Simulators  for  EW  Testing 

This  gap  in  test  capability  was  initially  identified  in  DOT&E ’s 
FY13  Annual  Report  due  to  the  continued  reliance  on  manned 
aircraft  for  captive-carry  of  the  ASCM  seeker  simulators.  Such 
simulators  will  be  unable  to  demonstrate  a  kinematic  response 
to  electronic  attack  by  SEWIP  Block  3  nor  demonstrate  the 
effect  such  kinematic  responses  will  have  on  ships’  hard-kill 
(e.g.  missiles,  guns)  systems.  Manned  aircraft  fly  too  high  and 
too  slowly  for  credible  ASCM  representation  and  are  unable  to 
represent  ASCM  maneuvers.  Credible  ASCM  representation 
requires  a  vehicle  that  can  fly  at  subsonic  ASCM  speeds 
and  lower  altitudes  than  the  current  Lear  Jets;  can  home  on 
a  platform  representing  a  SEWIP  Block  3-mounted  ship, 
using  a  threat-representative  radar  seeker  and  autopilot;  and 
can  respond  realistically  to  Block  3  electronic  jamming.  An 
approach  to  satisfy  this  requirement  is  a  recoverable,  unmanned 
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aerial  vehicle  using  embedded,  miniaturized  simulators  that 
can  maneuver  at  ASCM  speeds  and  altitudes  with  encrypted 
telemetry  to  track  seeker/autopilot  responses  to  electronic  attack. 
A  human-controlled  override  capability  would  be  required  for 
safe  operation.  The  remotely  controlled  Self-Defense  Test  Ship 
(SDTS)  would  tow  a  ship  target  for  the  unmanned  aerial  vehicles 
to  home  on.  SEWIP  Block  3  would  be  mounted  on  the  SDTS 
along  with  hard-kill  systems  such  that  the  integrated  hard-kill/ 
soft-kill  (i.e.  SEWIP  Block  3)  combat  system  capability  could 
be  demonstrated.  Currently,  such  testing  is  at  the  discrete 
combat  system  element  level,  leaving  integrated  combat  system 
capability  unknown. 

SEWIP  Block  3  IOT&E  is  projected  for  FY19.  FOT&E  of 
Block  3  integrated  with  the  DDG  1000  combat  system,  as  well 
as  FOT&E  with  the  CVN  78  combat  system,  should  occur 
subsequent  to  the  IOT&E.  The  cost  for  development  of  these 
unmanned  aerial  vehicles  (with  simulators  and  telemetry)  is 
estimated  to  be  approximately  $120.0  Million  for  development, 
testing,  and  acquisition.  Estimated  unit  cost  of  each  vehicle  is  not 
expected  to  exceed  $15.0  Million. 

Equipping  Self-Defense  Test  Ship  (SDTS)  for  Aegis  Combat 
System,  Air  and  Missile  Defense  Radar  (AMDR)  and  Evolved 
SeaSparrow  Missile  (ESSM)  Block  2  Operational  Testing 

The  close-in  ship  self-defense  battle  space  is  complex  and 
presents  a  number  of  challenges.  For  example,  this  environment 
requires: 

•  Weapon  scheduling  with  very  little  time  for  engagement 

•  The  necessity  of  the  combat  system  and  its  sensors  to  deal 
with  debris  fields  generated  by  successful  engagements  of 
individual  ASCMs  within  a  multi-ASCM  raid 

•  Rapid  multi-salvo  kill  assessments  for  multiple  targets 

•  Transitions  between  ESSM  guidance  modes 

•  Conducting  BMD  and  area  air  defense  missions  (i.e., 
integrated  air  and  missile  defense)  while  simultaneously 
conducting  ship  self-defense 

•  Contending  with  stream  raids  of  multiple  ASCMs  attacking 
along  the  same  bearing,  in  which  directors  illuminate  multiple 
targets  (especially  true  for  maneuvering  threats) 

•  Designating  targets  for  destruction  by  the  Close-In  Weapons 
System 

Multiple  hard-kill  weapons  systems  operate  close-in,  including 
the  Standard  Missile  2  (SM-2),  the  ESSM,  and  the  CIWS. 

Soft-kill  systems  such  as  Nulka  MK  53  decoy  launching  system 
also  operate  close-in.  The  short  timelines  required  to  conduct 
successful  ship  self-defense  place  great  stress  on  combat  system 
logic,  combat  system  element  synchronization,  combat  system 
integration,  and  end-to-end  performance. 

Navy  range  safety  restrictions  prohibit  close-in  testing  on  a 
manned  ship  because  the  targets  and  debris  from  successful 
intercepts  will  pose  an  unacceptable  risk  to  the  ship  and  personnel 
at  the  ranges  where  these  self-defense  engagements  take  place. 
These  restrictions  were  imposed  following  a  February  1983 
incident  on  the  USS  Antrim  (FFG  20),  which  was  struck  with  a 
subsonic  BQM-74  aerial  target  during  a  test  of  its  self-defense 


weapon  systems,  killing  a  civilian  instructor.  The  first  unmanned, 
remotely  controlled  SDTS  (the  ex-Stoddard)  was  put  into  service 
that  same  year.  A  similar  incident  occurred  in  November  2013, 
where  two  sailors  were  injured  when  the  same  type  of  aerial 
target  struck  the  USS  Chancellorsville  (CG  62)  during  what  was 
considered  to  be  a  low-risk  test  of  its  combat  system.  This  latest 
incident  underscores  the  inherent  dangers  of  testing  with  manned 
ships  in  the  close-in  battlespace. 

While  the  investigation  into  the  Chancellorsville  incident  has 
caused  the  Navy  to  rethink  how  they  will  employ  subsonic  and 
supersonic  aerial  targets  near  manned  ships,  the  Navy  has  always 
considered  supersonic  ASCM  targets  a  high  risk  to  safety  and 
will  not  permit  flying  them  directly  at  a  manned  ship.  The  Navy 
has  invested  in  a  current  at-sea,  unmanned,  remotely-controlled 
test  asset  (the  SDTS)  and  is  using  it  to  overcome  these  safety 
restrictions.  The  Navy  is  accrediting  a  high-fidelity  modeling 
and  simulation  (M&S)  capability  utilizing  data  from  the 
SDTS,  as  well  as  data  from  manned  ship  testing,  so  that  a  full 
assessment  of  ship  self-defense  capabilities  of  non-Aegis  ships 
can  be  completely  and  affordably  conducted.  While  the  Navy 
recognizes  the  capability  as  integral  to  the  test  programs  for 
certain  weapons  systems  (the  Ship  Self-Defense  System,  Rolling 
Airframe  Missile  Block  2,  and  ESSM  Block  1)  and  ship  classes 
(LPD  17,  LHA6,  Littoral  Combat  Ship,  LSD  41/49,  DDG  1000, 
and  CVN  78),  it  has  not  made  a  similar  investment  in  an  SDTS 
equipped  with  an  Aegis  Combat  System,  AMDR,  and  ESSM 
Block  2  for  adequate  operational  testing  of  the  DDG  5 1  Flight 
III  Destroyer  self-defense  capabilities.  The  current  SDTS  lacks 
the  appropriate  sensors  and  other  combat  system  elements  to  test 
these  capabilities. 

On  September  10,  2014,  DOT&E  submitted  a  classified 
memorandum  to  the  USD(AT&L)  with  a  review  of  the  Design 
of  Experiments  study  by  the  Navy  Program  Executive  Office 
for  Integrated  Warfare  Systems,  which  attempted  to  provide  a 
technical  justification  to  show  the  test  program  did  not  require 
an  SDTS  to  adequately  assess  the  self-defense  capability  of  the 
DDG  51  Flight  III  Class  Destroyers.  DOT&E  found  that  the 
study  presented  a  number  of  flawed  justifications  and  failed  to 
make  a  cogent  argument  for  why  an  SDTS  is  not  needed  for 
operational  testing. 

On  December  10,  2014,  the  Deputy  Secretary  of  Defense 
issued  a  memorandum  directing  the  Director,  Cost  Analysis/ 
Program  Evaluation  (CAPE)  to  identify  viable  at-sea  operational 
testing  options  that  meet  DOT&E  adequacy  requirements  and 
recommend  a  course  of  action  (with  cost  estimates,  risks,  and 
benefits)  to  satisfy  testing  of  the  AMDR,  Aegis  Combat  System, 
and  ESSM  Block  2  in  support  of  the  DDG  51  Flight  III  Destroyer 
program.  The  CAPE  study  evaluated  four  options  to  deliver 
an  at-sea  test  platform  adequate  for  self-defense  operational 
testing  of  the  DDG  51  Flight  III,  AMDR,  and  ESSM  Block  2 
programs.  Each  option  requires  funding  beginning  in  FY18  to 
ensure  support  of  operational  testing  of  these  systems  in  FY22. 
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A  decision  on  whether  to  fund  the  procurement  of  the  needed 
equipment  is  pending. 

DOT&E  continues  to  recommend  equipping  an  SDTS  with 
capabilities  to  support  Aegis  Combat  System,  AMDR,  and  ESSM 
Block  2  OT&E  to  test  ship  self-defense  systems’  performance  in 
the  final  seconds  of  the  close-in  battle  and  to  acquire  sufficient 
data  to  accredit  ship  self-defense  performance  M&S.  The 
CAPE-estimated  cost  for  development  and  acquisition  of 
these  capabilities  over  the  Future  Years  Defense  Program  is 
approximately  $350  Million.  Of  that,  approximately  half  could 
be  recouped  after  the  test  program  completes  by  installing 
the  hardware  in  a  future  DDG  51  Flight  III  Destroyer  hull. 

The  Navy  previously  agreed  with  this  “re-use”  approach  in 
their  December  2005  Air  Warfare/Ship  Self-Defense  Test  and 
Evaluation  Strategy  stating  that  .  upon  completion  of  testing 
and  when  compatible  with  future  test  events,  refurbish  and  return 
the  test  units  to  operational  condition  for  re-use.” 

Multi-Stage  Supersonic  Targets  (MSST) 

The  Navy  initiated  a  $297  Million  program  in  2009  to  develop 
and  produce  an  adequate  multi-stage  supersonic  target  (MSST) 
required  for  adequate  operational  testing  of  Navy  surface 
ship  air  defense  systems.  The  MSST  is  critical  to  the  DDG 
1000  Destroyer,  CVN  78  Aircraft  Carrier,  DDG  51  Flight  III 
Destroyer,  LHA(R),  AMDR,  Ship  Self-Defense  System,  Rolling 
Airframe  Missile  Block  2,  and  ESSM  Block  2  operational  test 
programs.  The  MSST  underwent  a  re-structure/re-baseline  from 
2013  -  2015  to  address  technical  deficiencies  as  well  as  cost 
and  schedule  breaches,  which  would  have  postponed  its  initial 
operational  capability  (IOC)  to  2020  and  increased  total  program 
cost  to  $962  Million.  Based  on  the  re-structured/re-baselined 
MSST  program’s  high  cost  and  schedule  delays  as  well  as  new 
intelligence  reports,  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition  in  2014  directed  that 
alternatives  be  examined  to  test  against  these  ASCM  threats 
and  subsequently  terminated  the  MSST  program.  While  the 
details  of  the  final  alternative  are  classified,  DOT&E  determined 
that  it  would  be  very  costly  (Navy  estimates  $739  Million), 
very  difficult  to  implement,  dependent  on  the  results  of  highly 
segmented  tests,  and  would  suffer  from  severe  artificialities  that 
would  hopelessly  confound  interpretation  of  test  results.  DOT&E 
informed  the  Navy  that  the  proposed  alternative  was  not  adequate 
for  operational  testing  and  recommended  that  the  Navy  not 
pursue  it. 

The  failure  of  the  MSST  program  and  the  inadequate  alternative 
proposal  is  perpetuating  poor  Fleet  understanding  of  how  well 
or  how  poorly  their  surface  combatants  will  be  able  to  defend 
themselves  against  MSST-like  ASCM  threats.  The  requirement 
for  a  viable,  cost-effective,  adequate  MSST  target  for  operational 
testing  remains  valid.  Nonetheless,  DOT&E  agrees  that 
terminating  the  failed  MSST  program  was  the  correct  decision. 

Fifth-Generation  Aerial  Target 

DOT&E  initiated  studies  in  2006  on  the  design  and  fabrication 
of  a  dedicated  fifth-generation  aerial  target  to  evaluate  U.S. 
weapon  systems  effectiveness.  The  study  team,  comprised  of 


Air  Force  and  Navy  experts,  retired  Skunk  Works  engineers, 
and  industry,  completed  a  preliminary  design  review  for  a 
government-owned  design.  DOT&E  requested  $27  Million  in 
the  FY17  program  review  to  complete  final  design,  tooling, 
and  prototyping  efforts.  The  prototyping  effort  will  provide 
cost-informed,  alternative  design  and  manufacturing  approaches 
for  future  air  vehicle  acquisition  programs.  This  data  can  also  be 
used  to  assist  with  future  weapon  system  development  decisions, 
T&E  infrastructure  planning/investment,  and  could  support  future 
analysis  of  alternative  activities.  The  prototype  design  directly 
supports  the  U.S.  Strategic  Command,  U.S.  Pacific  Command, 
and  U.  S.  Northern  Command's  Defense  Innovation  Initiatives  for 
persistent  cooperative  unmanned  aerial  systems  engagement. 

Warrior  Injury  Assessment  Manikin  (WlAMan) 

In  2010,  after  the  publication  of  the  DOT&E  survivability 
evaluation  of  the  MRAP  Family  of  Vehicles,  the  Secretary 
of  Defense  directed  an  evaluation  of  underbody  blast  (UBB) 
modeling  and  simulation  (M&S)  tools.  This  evaluation  was  to 
determine  if  an  enhanced  UBB  M&S  capability  could  identify 
potential  vulnerabilities  in  ground  combat  vehicle  designs 
while  still  in  the  early  stages  of  development.  The  evaluation 
identified  10  major  gaps  preventing  the  development  of  a 
comprehensive,  robust  UBB  M&S  capability  to  accurately  model 
the  effects  of  UBB.  The  top  three  gaps  were  all  associated  with 
the  shortcomings  in  available  instrumentation  and  criteria  to 
assess  human  injury  in  the  UBB  environment.  The  evaluation 
concluded  that  automotive  crash  test  dummies  used  in  LFT&E 
and  the  consequent  injury  criteria  designed  and  developed 
for  forces  and  accelerations  in  the  horizontal  plane  as  seen  in 
automotive  frontal  impact-induced  injuries  were  not  adequate  to 
assess  the  effects  of  the  forces  and  accelerations  in  the  vertical 
plane  typically  seen  in  combat-induced  UBB  events. 

In  2010,  DOT&E  submitted  an  issue  paper  advocating  the  need 
to  fund  the  identified  gaps  and  shortcoming  in  current  LFT&E 
practices.  This  led  to  the  Warrior  Injury  Assessment  Manikin 
(WlAMan)  project  with  an  $88  Million  budget  over  the  FY12-16 
Future  Years  Defense  Program.  Under  the  WlAMan  project, 
the  Army  initiated  critical  biomechanical  research  and  the 
anthropomorphic  test  devices  (ATD)  development  program  to 
increase  DOD’s  understanding  of  the  cause  and  nature  of  injuries 
incurred  in  UBB  combat  events. 

In  2013,  the  Army  created  a  dedicated  office  (the  WlAMan 
Engineering  Office  (WEO))  under  the  Army  Research, 
Development,  and  Engineering  Command  (RDECOM)  to 
manage  the  WlAMan  project.  In  2015,  the  office  of  the  Assistant 
Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology 
designated  the  WlAMan  project  as  an  Acquisition  Category 
II  acquisition  program  of  record  under  the  Program  Executive 
Office  for  Simulation,  Training,  and  Instrumentation  (PEO 
STRI).  PEO  STRI  and  RDECOM  finalized  the  WlAMan  Test 
Capabilities  Requirements  Document  and  a  formal  Program 
Office  Estimate  for  full  funding  of  the  program.  The  technical 
achievements  made  by  the  WEO  and  the  concerted  effort  by  the 
Army  to  create  the  foundation  for  a  formal  acquisition  program 
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represent  major  steps  forward  for  the  WIAMan  project,  and 
the  effort  is  poised  to  made  additional  progress  in  FY16  and 
beyond,  assuming  remaining  funding  is  allocated  to  allow  for  its 
completion. 

The  Assistant  Secretary  of  Defense  (Health  Affairs)  has 
committed  Science  &Technology  funding  to  the  program 
post-Milestone  B  to  ensure  critical  injury  biomechanics  research 
is  completed,  but  this  commitment  has  not  been  matched  by 
a  similar  commitment  from  the  Army  to  program  for  the  ATD 
production  and  procurement.  This  led  DOT&E  to  submit  another 
issue  paper  for  additional  funding  of  $98  Million  through  FY21 
that  would  enable  the  completion  of  research  and  development  of 
injury  criteria,  predictive  M&S,  and  development  of  the  technical 
data  package  including  two  generations  of  prototype  ATDs.  The 
Army  has  still  not  provided  funding  past  FY 17  jeopardizing  the 
continuity  and  completion  of  this  project. 

Some  within  the  Army  have  questioned  whether  DOD  still  needs 
a  combat-specific  injury  assessment  capability  but  in  the  view 
of  DOT&E,  it  is  entirely  appropriate  for  DOD,  and  in  particular 
for  the  Army,  to  accord  the  same  high  priority  to  testing  and 
verifying  the  protection  provided  to  Soldiers  by  their  combat 
vehicles  that  the  commercial  automotive  industry  accords  to 
testing  and  verifying  the  protection  provided  to  the  U.S.  public  by 
their  automobiles. 

Torpedo  Surrogates  for  Operational  Testing  of  Anti-Submarine 
Warfare  (ASW)  Platforms  and  Systems 

Operational  testing  of  ASW  platforms  and  related  systems 
includes  the  ability  to  detect,  evade,  counter,  and/or  destroy 
an  incoming  threat  torpedo.  The  determination  of  system  or 
platform  performance  is  critically  dependent  on  a  combination 
of  the  characteristics  of  the  incoming  torpedo  (e.g.,  dynamics, 
noise,  fusing,  sensors,  logic,  etc.).  Due  to  differences  in 
technological  approach  and  development,  U.S.  torpedoes  are  not 
representative  in  many  of  these  torpedo  characteristics  for  many 
highly  proliferated  torpedoes,  particularly  those  employed  in 
Anti-Surface  Warfare  by  other  nations.  Operational  testing  that  is 
limited  to  U.S.  exercise  torpedoes  will  not  allow  the  identification 
of  existing  limitations  of  ASW  systems  and  related  systems 
against  threat  torpedoes  and  will  result  in  uninformed  decisions 
in  the  employment  of  these  same  systems  in  wartime.  A 
January  9,  2013  DOT&E  memorandum  to  the  Assistant  Secretary 
of  the  Navy  for  Research,  Development,  and  Acquisition 
identifies  specific  threat  torpedo  attributes  that  the  threat  torpedo 
surrogate(s)  must  be  evaluated  against.  A  June  18,  2015 
DOT&E  memorandum  to  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition  reiterated  the  need  for 
representative  threat  torpedo  surrogates  in  operational  test  and 
emphasized  understanding  threat  torpedo  behavior,  including 
tactics  and  counter-measure  logic,  when  evaluating  adequacy  of 
torpedo  surrogates.  The  non-availability  of  threat-representative 
torpedo  surrogates  will  prevent  adequate  operational  testing  for 
ASW  platforms  and  related  systems,  as  well  as  adversely  affect 
tactics  development  and  validation  of  these  tactics  within  the 
fleet. 


Naval  Undersea  Warfare  Center  (NUWC)  Division  Keyport 
commenced  a  study  of  threat  torpedo  surrogates  in  FY14.  The 
$480,000  study  is  jointly  funded  by  the  Navy  and  DOT&E.  The 
completed  study,  dated  September  4,  2015,  confirmed  DOT&E 
concerns  that  current  torpedo  surrogates  have  significant  gaps 
in  threat  representation  for  operational  testing  and  provided 
recommendations  for  improving  current  threat  torpedo  emulation. 
However,  the  Navy  has  yet  to  provide  its  plan  for  adequate 
torpedo  surrogates  to  effectively  characterize  system  performance 
in  future  operational  tests. 

NUWC  Division  Keyport  is  pursuing  a  prototype  technology 
development  project  that  will  deliver  a  threat-representative 
high-speed  quiet  propulsion  system.  The  development  of  a 
propulsion  system  prototype  is  intended  to  overcome  a  critical 
gap  identified  in  the  torpedo  threat  surrogate  capability  gap 
analysis,  discussed  in  the  preceding  paragraph.  This  effort  is 
funded  by  DOT&E  at  approximately  $1.0  Million  with  delivery 
scheduled  in  4QFY16.  The  NUWC  Division  Keyport  study  and 
prototype  development  could  support  future  development  of  a 
threat  torpedo  surrogate.  Procurement  of  adequate  threat  torpedo 
surrogates,  however,  is  dependent  on  future  Navy  decisions. 

Submarine  Surrogates  for  Operational  Testing  of  Lightweight 
and  Heavyweight  Torpedoes 

The  Navy  routinely  conducts  in-water  operational  testing  of 
lightweight  and  heavyweight  ASW  torpedoes  against  manned 
U.S.  Navy  submarines.  Although  these  exercise  torpedoes  do 
not  contain  explosive  warheads,  peacetime  safety  rules  require 
that  the  weapons  run  above  or  below  the  target  submarine  with 
a  significant  depth  stratum  offset  to  avoid  collision.  While  this 
procedure  allows  the  torpedo  to  detect,  verify,  and  initiate  homing 
on  the  target,  it  does  not  support  assessment  of  the  complete 
homing  and  intercept  sequence.  One  additional  limitation  is  the 
fact  that  U.S.  nuclear  attack  submarines  may  not  appropriately 
emulate  the  active  target  strength  (sonar  cross-section)  of  smaller 
threats  of  interest,  such  as  diesel-electric  submarines.  During 
the  MK  50  lightweight  torpedo  operational  test  in  May  1992,  the 
Navy  conducted  some  limited  set-to-hit  testing  against  manned 
submarines,  which  included  impact  against  the  target  hull,  but 
that  practice  has  been  discontinued. 

In  preparation  for  the  2004  MK  54  lightweight  torpedo 
operational  test,  DOT&E  supported  the  development  and 
construction  of  the  unmanned  Weapon  Set-to-Hit  Torpedo  Threat 
Target  (WSTTT)  using  Resource  Enhancement  Project  (REP) 
funding.  The  WSTTT  was  a  full-sized  steel  mock-up  of  a  small 
diesel-electric  submarine,  with  an  approximate  program  cost  of 
$11  Million.  As  a  moored  stationary  target,  the  WSTTT  was 
limited  in  its  ability  to  emulate  an  evading  threat,  but  its  use  in 
the  MK  54  operational  test  demonstrated  the  value  of  such  a 
dedicated  resource.  Unfortunately,  the  Navy  did  not  properly 
maintain  the  WSTTT  and  abandoned  it  on  the  bottom  of  the  sea 
off  the  California  coast  in  2006.  In  subsequent  years,  the  Navy 
was  able  to  make  some  limited  use  of  the  WSTTT  hulk  as  a 
bottomed  target  for  torpedo  testing. 
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In  a  separate  effort,  the  Navy  built  the  Mobile  Anti-Submarine 
Training  Target  (MASTT),  designed  to  serve  as  a  full-sized  threat 
surrogate  for  use  in  training  by  surface  and  air  ASW  forces.  The 
Chief  of  Naval  Operations  initiated  the  program  in  2010  with  the 
goal  of  achieving  operational  capability  by  late  2011.  After  four 
years  and  an  expenditure  of  approximately  $15  Million,  the  Navy 
has  yet  to  use  the  MASTT  in  training  and  seems  to  be  on  the 
brink  of  abandoning  the  asset.  The  Navy  resisted  design  input 
from  the  operational  test  community  and  made  it  clear  that  the 
MASTT  was  not  intended  to  support  torpedo  testing. 

In  support  of  a  2010  Urgent  Operational  Need  Statement, 
the  Navy  funded  the  construction  of  the  Steel  Diesel-Electric 
Submarine  (SSSK),  a  full-sized  moored  set-to-hit  target 
consisting  of  an  open  steel  framework  with  a  series  of  corner 
reflectors  to  provide  appropriate  sonar  highlights.  Unfortunately, 
this  surrogate  does  not,  in  fact,  provide  a  realistic  sonar  signature. 
Nonetheless,  the  Navy  used  the  SSSK  as  a  target  for  the  MK  54 
torpedo  in  a  2011  Quick  Reaction  Assessment  and  2013  FOT&E. 
As  part  of  the  Test  and  Evaluation  Master  Plan  approval  for  the 
latter,  DOT&E  sent  a  memorandum  indicating  that  the  Navy  must 
develop  an  appropriate  mobile  target  to  support  future  MK  54 
testing. 

Since  early  2013,  DOT&E  has  participated  in  a  Navy  working 
group  attempting  to  define  the  requirements  for  a  mobile  set  to 
hit  torpedo  target.  The  group  has  identified  a  spectrum  of  options 
and  capabilities,  ranging  from  a  torpedo-sized  vehicle  towing 
a  long  acoustic  array  to  a  full-sized  submarine  surrogate.  At 
the  very  least,  the  target  is  expected  to  be  mobile,  autonomous, 
and  certified  for  lightweight  torpedo  set-to-hit  scenarios.  More 
advanced  goals  might  include  realistic  active  and  passive  sonar 
signatures  to  support  ASW  search  and  reactive  capability  to 
present  a  more  realistically  evasive  target.  Cost  estimates  range 
from  under  $10  Million  for  a  towed  target  to  over  $30  Million  for 
a  full-sized  submarine  simulator. 

Signature  Data  Collection  for  Infrared  (IR)  Guided  Surface  to 
Air  and  Hostile  Fire  Threats  to  Support  Model  Development 

Threat  M&S  capabilities  are  essential  for  testing  missile  warning 
and  countermeasure  systems  under  development.  However, 
models  for  1R  guided  surface  to  air  and  unguided  threat  weapons 
do  not  adequately  represent  the  threat  characteristics  for  testing 
modern  missile  warning  systems  and  are  deficient.  To  support 
threat  model  development,  an  integrated,  transportable  capability 
to  measure  and  record  high  fidelity  signature,  Time  Space 
Position  Information,  and  related  information  for  live  fire  testing 
of  threat  missile  and  hostile  fire  munitions  (e.g.,  small  arms  and 
RPG)  firings  was  ranked  as  a  high  priority  need  by  the  Infrared 
Countermeasures  Test  Resource  Requirements  Study  (ITRRS) 
team  and  the  Threat  M&S  Roadmap.  Additionally,  the  Aircraft 
Survivability  Equipment  (ASE)  Program  Offices  from  each 
Service  have  endorsed  this  need  for  assessment  of  ground  truth, 
anomaly  resolution,  and  to  enhance  M&S  capabilities  for  the 
development  and  T&E  of  aircraft  self-defense  systems. 

DOT&E  supports  the  use  of  common,  authoritative  threat  M&S 
capabilities  for  ASE  testing.  For  example,  the  DOT&E  Center 


for  Countermeasures  serves  as  the  executing  activity  for  a 
Test  Resources  Management  Center  (TRMC)  Central  Test  and 
Evaluation  Investment  Program  (CTE1P)  REP,  known  as  Joint 
Standards  Instrumentation  Suite  (JSIS).  When  available,  the  JSIS 
IOC  will  support  Advanced  Threat  Warner  and  Department  of  the 
Navy  (DoN)  Large  Aircraft  Infrared  Countermeasure  (LAIRCM) 
operational  testing.  JSIS  can  be  deployed  to  OCONUS  static  live 
fire  venues  where  opportunities  exist  to  measure  and  collect  data 
for  threat  assets  that  are  either  not  available,  or  of  insufficient 
quantities  domestically.  JSIS  data  will  support  improvements 
to  existing  threat  models  and  help  create  models  of  new  threats. 
JSIS  will  provide  a  capability  for  use  by  each  Service  and  support 
other  operational  testing  needs. 

However,  the  JSIS  IOC  capability  only  partially  addresses  the 
needs  identified  by  the  ITRRS  team.  For  example,  it  will  not 
provide  the  capability  to  measure  missile  attitude  information  for 
the  entire  missile  flyout,  nor  will  the  JSIS  IOC  capability  meet 
all  needs  related  to  signature  collection  fidelity  (i.e.,  frame  rates 
and  resolution).  Full  operational  capability  is  required  to  meet 
the  needs  of  the  Army's  Common  Infrared  Countermeasures 
(CIRCM)  program,  Navy's  Advanced  Threat  Warning,  Air  Forces' 
LAIRCM  program,  and  Navy  Research  Laboratory's  Distributed 
Aperture  Infrared  Countermeasure  (DAIRCM)  program.  JSIS 
requires  an  additional  investment  of  $25  Million  to  provide  the 
full  operational  capability  needed  for  Infrared  Countermeasures 
(IRCM)  T&E. 

Threat  Modeling  and  Simulation  (M&S)  to  Support  Aircraft 
Survivability  Equipment  (ASE)  Testing 

Acquiring  actual  threat  systems  for  widespread  testing  may  not 
be  possible.  To  address  this  challenge,  DOT&E  has  funded 
standard,  authoritative  threat  M&S  for  systems  T&E.  In  some 
cases,  threat  M&S  used  in  T&E  have  not  provided  accurate 
representations,  and  different  M&S  instantiations  of  the  same 
threats  often  produced  different  results.  DOT&E’s  objective  is 
to  improve  the  fidelity  and  consistency  of  threat  M&S  at  various 
T&E  facilities  while  reducing  overall  test  costs. 

Throughout  the  T&E  process,  M&S  representations  of  threat 
systems  can  be  used  when  actual  threat  components  are  not 
available.  M&S  can  provide  a  more  complete  testing  capability 
than  possible  through  open-air  facilities  alone.  It  supports  testing 
when  flight  safety  precludes  live  testing,  such  as  missile  launches 
against  manned  aircraft.  Threat  M&S  may  be  used  to  extend  the 
results  of  live  missile  test  events  across  a  broader  range  of  test 
conditions,  with  different  threats,  ranges,  altitudes,  aspect  angles, 
atmospheric  conditions,  and  other  environmental  variables 
affecting  weapon  system  performance. 

DOT&E  has  a  T&E  Threat  M&S  Configuration  Management 
System  to  implement  controls  and  distribution  management 
for  threat  M&S.  This  Configuration  Management  System 
ensures  integrity  and  consistency  of  test  results  among  various 
T&E  M&S  regimes.  This  system  also  provides  mechanisms  to 
identify  and  correct  anomalies  between  a  threat  and  its  M&S 
representations.  It  assists  in  controlling  model  configuration 
changes,  maintaining  critical  documentation  such  as  interface 
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descriptions  and  validation  documents,  and  sharing  updated 
threat  M&S  with  multiple  T&E  facilities.  The  T&E  Threat  M&S 
Configuration  Control  Board,  comprised  of  representatives  from 
the  T&E  community  and  intelligence  organizations,  prioritizes 
existing  threat  M&S  developments  and  changes  to  ensure  updates 
are  provided  efficiently  to  T&E  user  facilities.  Requests  for  T&E 
threat  M&S,  anomaly  reports,  and  change  requests  are  managed 
by  DOT&E. 

During  FY15,  the  T&E  Threat  Resource  Activity  provided 
standardized  authoritative  threat  M&S  to  multiple  T&E  facilities 
operated  by  the  Army,  Navy,  and  Air  Force  in  support  of  Aircraft 
Survivability  Equipment  (ASE)  testing.  DOT&E  has  engaged 
our  closest  allied  nations  in  implementing  the  same  authoritative 
threat  M&S  for  allied  T&E.  This  allows  the  U.S.  and  its  allies  to 
efficiently  leverage  each  other’s  ranges  and  facilities. 

DOT&E  developed  and  updated  a  Threat  M&S  Roadmap  for 
ASE  T&E  to  provide  a  comprehensive  plan  for  future  threat 
M&S.  A  good  example  is  JSIS,  which  will  capture  threat  data 
from  live  test  events.  The  Roadmap  identifies  projects  to  conduct 
systematic  analyses  of  the  JSIS  data  to  feed  the  development 
of  threat-representative  M&S  to  support  U.S.  and  allied  missile 
warning  and  infrared  countermeasure  systems. 

Foreign  Materiel  Acquisition  Support  for  T&E 

DOT&E  is  responsible  for  ensuring  U.S.  weapons  systems  are 
tested  in  realistic  threat  environments  using  actual  threat  systems 
to  create  these  threat  environments  whenever  possible  and 
appropriate.  DOT&E  develops  an  annual  prioritized  list  of  threat 
requirements  tied  to  upcoming  testing  of  programs.  This  list  is 
submitted  it  to  the  DIA  Joint  Foreign  Materiel  Program  Office. 
These  requirements  are  consolidated  with  Service  needs  and  then 
processed  through  various  Service  and  intelligence  community 
collection  activities.  DOT&E  coordinates  with  the  Department 
of  State  to  identify  resource  providers  to  increase  opportunities  to 
acquire  foreign  materiel  for  use  in  OT&E. 

Foreign  materiel  requirements  span  all  warfare  areas,  but 
DOT&E  continues  to  place  a  priority  on  the  acquisition  of 
Man-Portable  Air  Defense  Systems  (MANPADS)  to  address 
significant  threat  shortfalls  that  affect  testing  for  IRCM  programs 
like  CIRCM,  LAIRCM,  and  DoN  LAIRCM.  In  some  programs, 
a  large  number  of  MANPADS  are  required  for  development  of 
threat  M&S,  for  use  in  hardware-in-the-loop  laboratories,  and  for 
LFT&E,  to  present  realistic  threats  to  IRCM  equipment.  Using 
actual  missiles  and  missile  seekers  aids  evaluators  in  determining 
the  effectiveness  of  IRCM  equipment.  This  past  year,  several 
ongoing  Foreign  Material  Acquisition  efforts  have  led  to  new 
opportunities  to  acquire  IRCM  equipment. 

When  acquiring  specific  hardware  is  not  possible,  the  acquisition 
of  technical  documentation  may  be  possible.  Evaluating 
technical  documentation  is  valuable  because  it  supports  the 
development  of  specific  threat  simulators  to  be  used  at  T&E 
ranges  and  facilities. 

Due  to  the  inherent  challenge  of  developing  reliable  sources  for 
foreign  materiel,  negotiating  the  acquisition  of  foreign  materiel, 


and  the  difficulty  of  using  annual  appropriations  for  foreign 
materiel  acquisitions,  DOT&E  supports  the  establishment 
of  dedicated,  non-expiring  funding  authority  within  the 
DOD  Foreign  Materiel  Program  to  support  foreign  materiel 
acquisitions. 

Tactical  Engagement  Simulation  with  Real  Time  Casualty 
Assessment  (TES/RTCA ) 

Realistic  operational  environments  and  a  well-equipped  enemy 
intent  on  winning  are  fundamental  to  the  adequate  operational 
test  of  land  and  expeditionary  combat  systems.  Force-on-force 
battles  between  tactical  units  represent  the  best  method  of 
creating  a  complex  and  evolving  battlefield  environment  for 
test  and  training.  Simulated  force-on-force  battles  must  contain 
realism  to  cause  commanders  and  Soldiers  to  make  tactical 
decisions  and  react  to  the  real-time  conditions  on  the  battlefield. 
TES/RTCA  systems  integrate  live,  virtual,  and  constructive 
components  to  enable  these  simulated  force-on-force  battles, 
and  provide  a  means  for  simulated  engagements  to  have  realistic 
outcomes  based  on  the  lethality  and  survivability  characteristics 
of  both  the  systems  under  test  and  the  opposing  threat  systems. 
TES/RTCA  systems  must  replicate  the  critical  attributes  of 
real-world  combat  environments  such  as  direct  and  indirect  fires, 
IEDs  and  mines,  realistic  battle  damage,  and  casualties.  TES/ 
RTCA  systems  must  record  the  time-space  position  information 
and  firing,  damage,  and  casualty  data  for  all  players  in  the  test 
event  as  an  integrated  part  of  the  test  control  and  data  collection 
architecture.  Post-test  playback  of  these  data  provides  a  critical 
evaluation  tool  to  determine  the  combat  system’s  capability  to 
support  Soldiers  and  Marines  as  they  conduct  combat  missions. 

DOT&E  has  recommended  the  Army  Test  and  Evaluation 
Command  (ATEC)  and  the  Marine  Corps  Test  and  Evaluation 
Activity  (MCOTEA)  leverage  existing  TES/RTCA  capabilities 
to  support  upcoming  operational  tests  and  make  necessary 
investments  to  meet  known  capability  shortfalls  and  future 
requirements.  Shortfalls  include  the  ability  to  seamlessly 
simulate  indirect  fire  weapons,  IEDs/mines,  and  air-to-ground/ 
ground-to-air  combat  including  manned  and  unmanned  teaming. 
Future  requirements  include  new  and  upgraded  combat  vehicles, 
expanded  use  of  remote  weapon  stations,  and  evolving  threat 
systems. 

In  FY15,  the  Army  increased  their  planned  funding  for  the 
Integrated  Test  Live,  Virtual,  and  Constructive  Environment 
(ITLE)  project,  which  was  created  to  address  the  known  TES/ 
RTCA  capability  shortfalls  and  future  Army  requirements. 

ITLE  will  adapt  and  integrate  a  number  of  currently  disparate 
capabilities  and  take  advantage  of  recent  investments  made  by 
the  Army  training  community.  DOT&E  is  encouraged  by  the 
increase  in  dedicated  TES/RTCA  resources  and  the  continued 
cooperation  between  the  test  and  training  communities  in  the 
Army.  Beginning  in  FY16,  ATEC  is  working  to  resolve  issues 
with  its  airborne  TES/RTCA  capability  in  support  of  upcoming 
operational  tests  of  the  Apache,  Gray  Eagle,  and  Shadow 
manned/unmanned  teaming  capability.  Funding  for  this  upgrade 
was  anticipated  to  be  provided  by  the  CTEIP  REP,  but  was 
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diverted  to  other  higher  priority  efforts.  DOT&E  continues  to 
support  CTEIP  and  ATEC  funded  efforts  to  provide  this  needed 
capability. 

The  Marine  Corps’  current  force-on-force  training  system,  the 
Instrumented  Tactical  Engagement  Simulation  System  II,  does 
not  support  combat  vehicle  engagements.  MCOTEA  had  planned 
a  substantial  upgrade  beginning  in  FY16  to  support  the  upcoming 
operational  testing  of  the  Amphibious  Combat  Vehicle  and 
Amphibious  Assault  Vehicle  -  Survivability  Upgrade  programs. 
Funding  for  this  upgrade  was  anticipated  to  be  provided  by  the 
CTEIP  REP,  but  was  diverted  to  other  higher  priority  efforts. 
DOT&E  continues  to  support  CTEIP  and  MCOTEA  funded 
efforts  to  provide  this  needed  capability. 

TES/RTCA  capabilities  are  essential  for  realistic  force-on-force 
testing  of  current  and  future  land  and  expeditionary  warfare 
systems;  DOT&E  requires  these  capabilities  for  systems  such  as 
Amphibious  Combat  Vehicle,  Bradley  and  Abrams  Upgrades, 
Armored  Multi-purpose  Vehicle,  AH-64E  Block  III,  Joint  Light 
Tactical  Vehicle,  and  Stryker  Upgrades. 

Testing  in  Urban  Environments 

Operations  in  urban  environments  present  unique  challenges  to 
the  military  Services  and  their  equipment.  Degraded  mobility, 
communications,  and  situational  awareness;  a  large  civilian 
presence;  the  risk  of  collateral  damage;  reduced  stand-off 
distances;  and  unique  threat  profiles  are  some  of  the  conditions 
present  during  urban  operations.  These  challenges  and  a  world 
population  that  is  becoming  increasingly  urban,  reinforce  the 
requirement  that  systems  conduct  operational  testing  in  realistic 
urban  environments. 

From  2009  to  2011,  the  Army  conducted  the  Urban  Environment 
Test  Capability  study  that  collected  data  on  cities  around  the 
world  and  characterized  aspects  of  urban  environments  important 
to  military  operations.  The  Urban  Environment  Test  Capability 
final  report  was  used  to  support  a  Test  Capabilities  Requirements 
Document  for  the  Army  led  Joint  Urban  Test  Capability  (JUTC) 
project.  The  JUTC  planned  to  build  a  reconfigurable  urban  area 
with  modular  structures  from  one  to  five  stories  tall  on  the  White 
Sands  Missile  Range  (WSMR),  New  Mexico.  The  JUTC  began 
design  and  development  efforts  in  1QFY12,  but  was  canceled  in 
2QFY15  due  to  programmatic  delays,  a  de-scoping  of  the  original 
requirement,  and  cost  growth. 

The  result  of  the  cancellation  of  JUTC  is  that  the  long-standing 
urban  environment  operational  and  developmental  test  capability 
shortfall  has  not  been  addressed.  DOT&E  recommends  that  the 
Army  focus  research  funding  on  the  fundamental  engineering 
challenges  of  producing  an  affordable  structure  concept  that 
could  be  applied  not  only  at  WSMR,  but  also  on  other  test  and 
training  ranges  where  operational  tests  are  conducted.  The 
JUTC  Test  Capability  Requirement  should  be  revisited  to  capture 
current  T&E  requirements  and  future  efforts  should  take  into 
consideration  the  lessons  learned  from  the  failure  of  JUTC. 


Biological  Defense  Testing  at  West  Desert  Test  Center  on 
Dugway  Proving  Ground 

In  late  FY15,  DOD  suspended  the  production  of  and  testing 
with  biological  select  agents  and  toxins  (BSAT)  and  derivatives 
of  BSAT  materials  at  Dugway  Proving  Ground  pending  an 
investigation  and  review  of  safety  and  surety  protocols  and 
procedures.  The  suspension  has  temporarily  imposed  limitations 
to  DOD’s  ability  to  test  and  evaluate  biological  defense  systems. 
As  directed  by  Deputy  Secretary  of  Defense,  a  Biosafety  Task 
Force  is  reviewing  all  DOD  activities  engaged  in  handling  BSAT 
and  providing  recommendations  to  ensure  the  safety  and  surety 
of  DOD  protocols  and  procedures.  The  West  Desert  Test  Center 
Life  Sciences  Division  will  be  required  to  implement  improved 
biosafety  and  surety  protocols  and  procedures  before  seeking 
Centers  for  Disease  Control  and  Prevention  certification  to 
operate  at  Bio  Surety  Level  Three  to  resume  full  test  capabilities. 
West  Desert  Test  Center  has  unique  biological  testing  facilities 
that  provide  operationally  realistic  T&E  of  biological  defense 
systems. 

Range  Sustainability 

Adequate  mission  space  to  conduct  operationally  realistic 
testing  on  DOD’s  air-land-sea  test  and  training  ranges  is  a 
critical  resource  for  developing  weapons  systems  that  are 
effective,  reliable,  and  lethal.  DOD  test  and  training  ranges 
face  environmental  and  mission  compatibility  encroachment 
challenges  that,  if  not  resolved  successfully,  will  adversely  affect 
test  capabilities.  Accordingly,  DOT&E  continues  efforts  on 
behalf  of  the  T&E  community  to  assess,  mitigate  where  possible, 
and  resolve  compatibility  challenges  so  that  DOD’s  mission 
space  is  preserved  for  operationally  realistic  testing. 

DOT&E  is  focusing  on  improvements  to  compatibility  evaluation 
processes,  so  that  deficiencies  can  be  addressed  promptly,  and 
with  analytical  rigor  and  documentation  to  support  decision 
makers.  The  continuing  major  areas  of  concern  for  compatibility 
evaluations  are: 

•  Wind  energy  and  transmission  line  projects 

•  Outer  Continental  Shelf  (OCS)  oil  and  gas  leasing 

•  Foreign  investment 

•  Threatened  and  endangered  species 

Wind  energy  projects,  can  adversely  affect  testing  capabilities  by 
interfering  with  test  range  radars  and  datalinks.  DOD  receives, 
on  average,  66  such  projects  a  month  for  evaluation  of  risk  to 
mission  capabilities.  A  significant  improvement  in  the  DOD 
evaluation  process  in  2014  resulted  in  more  timely  and  effective 
consideration  of  projects  undergoing  review.  For  example, 
where  a  wind  turbine  project  was  found  to  have  the  potential 
to  seriously  degrade  radar  cross  section  testing  at  the  Naval  Air 
Warfare  Test  Center,  Patuxent  River,  Maryland,  a  timely  DOD 
objection  on  the  basis  of  Adverse  Impact  to  National  Security 
was  filed  with  the  Department  of  Transportation  based  on  a 


406  T&E  Resources 


F  Y 1  5  TEST  AND  EVALUATION  RESOURCES 


thorough  evaluation  by  DOD,  and  the  developer  subsequently 
withdrew  the  application  for  the  project. 

There  has  been  an  increase  of  over  20  percent,  between  2014 
and  2015,  in  the  number  of  transmission  line  projects  referred  to 
DOD  for  review  for  compatibility  concerns.  DOT&E  actively 
participated  in  the  review  of  these  projects,  and  coordinated  its 
evaluations  with  those  of  other  DOD  components.  In  the  case 
of  the  SunZia  transmission  line  project,  DOT&E-led  test-related 
reviews  determined  that  the  proposed  line  routing  would  impair 
networked  missile  intercept  testing  at  WSMR.  DOD  reached 
an  agreement  with  the  Bureau  of  Land  Management  to  bury 
portions  of  the  transmission  lines  in  areas  most  critical  for 
missile  intercept  testing.  Subsequently,  DOT&E  conducted  a 
post-decision  SunZia  lessons  learned  study  intended  to  help 
improve  DOD  evaluation  processes,  and  to  include  more 
effective  interaction  with  other  federal  agencies. 

DOT&E  continues  to  work  with  the  office  of  the  Assistant 
Secretary  of  Defense  (Readiness)  to  coordinate  the  DOD 
response  to  the  Bureau  of  Ocean  Energy  Management  on 
proposed  oil  and  gas  lease  plans  for  the  OCS.  Areas  considered 
for  such  leases  are  often  the  same  areas  where  DOD  testing  must 
be  conducted.  Continued  use  of  these  areas  is  critical  so  that  test 
realism  is  achieved  and  public  safety  is  preserved.  Consequently, 
DOT&E  is  engaged  in  evaluating  test  capability  risk  from 
proposed  leaseholds  in  the  OCS  and  representing  those  risks  in 
developing  the  Bureau  of  Ocean  Energy  Management  2017  to 
2022  lease  plan  so  that  weapons  system  testing  requirements  are 
balanced  with  national  energy  needs. 

Foreign  investment  in  the  United  States  near  test  ranges  is  a  new 
concern  due  to  possible  security  risks  for  foreign  data  collection. 
Recognizing  this  concern,  DOD  refers  some  20  projects  per 
month  from  the  Congressional  Committee  on  Foreign  Investment 
in  the  United  States  to  DOT&E  for  evaluation.  An  analysis 
methodology,  developed  by  DOT&E,  is  being  used  to  determine 
whether  there  are  potential  risks  to  test  resources. 

Species  and  habitat  environmental  concerns  continue  to  be  issues 
for  test  ranges.  There  are  145  candidate  species  now  awaiting 
U.S.  Fish  and  Wildlife  Service  listing  determinations,  including 
25  species  which  could  potentially  impact  military  test  and 
training.  To  ensure  a  balance  of  testing  requirements  with  species 
protection,  DOT&E  monitors  potential  impacts  to  test  ranges.  In 
collaboration  with  other  DOD  and  Federal  Agencies,  DOT&E 
continues  to  seek  proactive  solutions  that  will  minimize  negative 
impacts  for  use  of  range  space. 

DOT&E’s  range  sustainability  work  also  relies  on  outreach 
with  regional  partnerships  to  include  the  Southeast  Regional 
Partnership  for  Planning  and  Sustainability,  Western  Regional 
Partnership,  Land  Trust  Alliance,  other  Federal  agencies,  the 
Range  Commanders  Council,  and  Service  Program  Executive 
Offices.  This  outreach  provides  a  mechanism  for  mutual 
understanding  of  DOD  and  external-to-DOD  requirements  in 
addressing  range  sustainability  issues.  This  outreach  enables 
DOD  to  educate  external  organizations  on  why  resources  are 


needed  for  test  purposes,  and  at  the  same  time  gives  DOD 
improved  access  to,  and  awareness  into,  external-to-DOD 
information  and  processes. 

Continuing  Radio  Frequency  Spectrum  Concerns 

Adequate  frequency  spectrum  is  a  critical  resource  for 
testing.  It  is  required  to  both  upload  and  download  test  data 
between  the  article  being  tested  to  test  instrumentation,  and 
to  control  resources  during  test  operations.  At  the  World 
Radiocommunication  Conference  2007  (WRC-07),  the  United 
States  position  was  that  there  is  a  large  and  growing  shortfall 
of  global  or  regional  Aeronautical  Mobile  Telemetry  (AMT) 
allocations.  With  increasing  data  rates  associated  with  the  testing 
of  new  and  emerging  technologies,  the  United  States  believed 
that  an  additional  650  Megahertz  (MHz)  would  be  required  for 
AMT. 

Test  range  use  of  frequency  spectrum  continues  to  be  challenged 
by  pressures  to  repurpose  spectrum  to  broadband  wireless  and 
to  support  emerging  technologies  such  as  small  unmanned 
airborne  systems.  With  domestic  and  international  spectrum 
being  repurposed  for  non-defense  wireless  transmission 
needs,  DOT&E  remains  actively  engaged  with  the  DOD  Chief 
Information  Officer,  Deputy  Assistant  Secretary  of  Defense 
(Developmental  Test  and  Evaluation),  and  TRMC,  to  ensure 
that  frequency  spectrum  allocations  are  sufficient  for  the 
conduct  of  test  operations,  and  also  that  these  operations  use 
frequency  efficiently.  This  spectrum  efficiency  goal  is  being 
actively  pursued  through  the  TRMC  administered  Science  and 
Technology  program  and  CTEIR 

DOT&E  documented  the  pending  loss  of  the  1,755  -  1,780  MHz 
band  and  compression  into  1,780  -  1,850  MHz  in  its  FY13 
Annual  Report.  This  loss  occurred  during  the  Advanced  Wireless 
Services  -  3  auction,  which  concluded  January  29,  2015.  The 
impacts  to  the  Services’  T&E  infrastructure  for  transitioning 
AMT  capabilities  from  this  spectrum  in  the  L-band  are: 

•  Army  T&E  requires  ~  $27.7  Million  to  retrofit  Aerial 
Telemetry  Systems  at  WSMR  and  to  compress  operations  into 
the  1,780  -  1,850  MHz  band.  An  additional  $1.0  Million  is 
required  to  replace  point-to-point  datalinks  at  Aberdeen  Test 
Center,  Aberdeen  Proving  Ground,  Maryland.  Testing  of 
robotics  will  be  relocating  to  4  Gigahertz  (C-band),  which  will 
require  new  equipment  to  be  installed. 

•  Navy  T&E  requires  ~  $108  Million  to  compress  AMT 
operations  into  the  1,780  -  1,850  MHz  band  and  to  make  smart 
investments  in  ground  and  airborne  infrastructure  to  utilize 
C-band  AMT  frequencies  where  practicable.  In  accordance 
with  the  National  Telecommunications  and  Information 
Administration,  Federal  Communications  Commission  (FCC) 
and  Office  of  Management  of  Budget  approved  transition  plan, 
the  Navy  will  modify  ground  and  airborne  AMT  systems, 
including  incorporating  more  efficient  telemetry  modulation 
techniques,  adding  multi-band  antennas,  and  installing 
interference-monitoring  equipment.  The  Navy  transition 
plan  also  accounts  for  Missile  Defense  Agency  (MDA) 
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requirements.  Timelines  for  transition  range  from  36  months 
(MDA)  to  102  months,  depending  on  the  installation.  To 
minimize  impacts  on  operational  military  mission  capabilities, 
the  Navy  will  also  purchase  five  mobile/transportable 
telemetry  units  to  supplement  capacity  while  AMT  receiver 
sites  are  offline  for  modification. 

•  Air  Force  T&E  requires  ~  $348  Million  to  compress  into  the 
1,780  -  1,850  MHz  band.  The  funds  are  required  to  modify 
ground  and  airborne  systems,  including  incorporating  more 
efficient  modulation  techniques,  adding  multi-band  antennas, 
and  installing  interference-monitoring  equipment.  Timelines 
for  this  transition  range  from  66  to  120  months,  depending 
on  the  installation.  To  minimize  impacts  on  operational 
military  mission  capabilities,  the  Air  Force  will  also  purchase 
six  mobile/transportable  telemetry  units  to  supplement 
capacity  while  AMT  receiver  sites  are  offline  for  modification. 

Table  2  illustrates  the  frequency  bands  used  for  T&E,  and 
identifies  resource  deficiencies  and  their  potential  mitigations. 

As  the  table  below  points  out,  both  the  range’s  primary  L-  and 
S-bands  have  been  identified  for  study  to  support  the  National 
Broadband  plan,  published  in  March  2010  whereby  500  MHz 
would  be  repurposed  from  federal  and  non-federal  bands  for 
broadband  wireless  use.  The  spectrum  now  allocated  to  test  is 
used  full  time  during  the  range  day  (i.e.,  from  6:00am  to  6:00pm), 
and  continued  unimpeded  use  is  critical  to  allow  for  collection  of 
the  increasing  volume  of  test  data  (e.g.,  that  of  the  F-35  JSF). 

The  test  ranges’  are  currently  working  two  problems  in  the 
primary  band  for  telemetry,  1,435  -  1,525  MHz: 

1.  The  first  problem  is  the  recently  approved  FCC  rulemaking 
to  allow  sharing  of  the  spectrum  with  wireless  microphones 
used  for  major  concerts  and  sports  events.  DOD  has  worked 
successfully  with  industry  to  adopt  the  use  of  agreements 
(such  as  not-to-interfere  agreements)  and  electronic  keys  to 
coordinate  band  usage.  However,  the  development  of  the 
electronic  key  technology  has  not  been  done  and  its  reliability 
has  not  been  demonstrated. 


2.  The  second  problem  has  greater  potential  impact  to  the  test 
community  and  stems  from  proposed  WRC  repurposing  of 
AMT  allocated  spectrum  for  worldwide  wireless  broadband 
use,  which  both  Canada  and  Mexico  support.  The  United 
States  has  notified  its  neighbors  it  intends  to  continue  using 
the  band  for  telemetry  albeit  in  accordance  with  any  protection 
agreements  concluded  with  each  neighbor.  Due  to  the  location 
of  many  of  the  test  ranges  in  the  Southwest  continental 
United  States  and  commercial  aircraft  manufacturers’  testing 
proximate  to  the  U.S.  and  Canadian  border,  repurposing  of 
the  1,435  -  1,525  MHz  spectrum  for  wireless  broadband  is 
of  major  concern  due  to  its  potential  to  interfere  with  AMT 
operations.  Canada  has  engaged  with  DOD  and  the  aircraft 
industry  to  define  protection  criteria  for  both  U.S.  and 
Canadian  systems  to  take  effect  when  Canada  begins  using  the 
band  for  wireless  broadband  service.  Mexico  has  also  been 
approached  to  work  mitigation  strategies  for  the  same  reason. 

The  second  most-used  band  for  test  range  telemetry  is  the 
2,360  -  2,390  MHz  spectrum.  Again  the  issue  confronting  the 
ranges  is  the  potential  interference  with  AMT  operations  from 
assignment  of  adjacent  spectrum  (2,345  -  2,360  MHz)  to  wireless 
broadband  use.  The  vendor  for  operations  in  this  spectrum  has 
agreed  to  use  of  the  International  Telecommunications  Union 
recommendation  that  prescribes  out-of-band  emissions  protection 
for  telemetry  systems.  DOD  continues  to  work  this  issue  with 
both  the  FCC  and  the  vendor. 

Frequency  spectrum  is  a  limited  resource  with  many  more 
demands  than  available  supply.  The  DOD  published  its 
Electromagnetic  Spectrum  Strategy  at  the  end  of  2013,  followed 
by  the  Roadmap  and  Action  Plan  that  will  guide  the  strategy 
implementation  in  2015.  A  major  element  of  the  strategy  is 
an  emphasis  on  spectrum  sharing  vice  spectrum  reallocation, 
because  both  DOD  and  the  private  industry  sector  demands  are 
growing  at  rapidly. 
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TABLE  2.  FREQUENCY  ALLOCATIONS  USED  FORTESTING  AND  DOD  RESOURCE  ISSUES  AND  POTENTIAL  MITIGATIONS 


Frequency 

Use 

Users 

Resource  Issue  and  Potential  Mitigation 

Notes 

406.1  -420  MHz 

Land  mobile  radio 

Test  control  and  field  operators 

1350 -1390  MHz 

Time,  Space, 
Position 

Information 

Critical  to  almost  all  open-air  tests; 
range  surveillance  radar  (Air  Route 
Surveillance  Radar-4) 

Band  is  part  of  1 300-1400  MHz  band 
under  consideration  for  reallocation  to 
broadband  at  WRC-1 5.  US  will  declare  a 
no-change  position  if  it  comes  to  fruition, 
but  will  need  to  constrain  operations  along 
boarders  if  Mexico  and/or  Canada  adopt 
such  a  change. 

Band  is  where  position  location 
systems  (TSPI)  operate.  Used  at  most 
test  ranges,  some  training  ranges. 

1435 -1525  MHz 

L-Band  Telemetry  - 
Primary  Telemetry 
Band 

SDB,  UH1/AH,T-45,  SH-60,  VH-S,V-22, 
F-1 8,  F-1 8E,  F-22,  F-35,  B-2,  F-1 6,  B-1 , 
B-2,  B-52,  Global  Hawk 

•  Issue:  Wireless  microphone  use. 

•  Potential  Mitigation:  Alternate  user 
coordination  with  assigned  key  codes  for 
spectrum  access  in  allotted  time  periods. 

•  Issue:  WRC  assignment  to  worldwide 
wireless  broadband  use. 

•  Potential  Mitigation:  Ongoing 
negotiations  with  Canada  and  Mexico. 

Regardless  of  outcomes  of  Canada  & 
Mexico  negotiations,  usage  would  still 
be  constrained  along  borders. 

1675 -1710  MHz 

Weather,  including 
wind  speed 
measurement 

Critical  to  almost  all  open-air  tests 

1755 -1780  MHz 

L-Band  Telemetry 

F/EA-1 8G,  Aerostar,  ASVS,  SM-2,  RAM, 
SSRT,  Classified  UAV  (WSMR),  ARAV, 
X-47,  the  only  band  for  miss-distance 
indicators  used  to  score  missile  shots 

•  Issue:  Advanced  Wireless  Services  -  3 
auction  completed. 

•  Mitigation:  Use  compression  and 
relocation  to  4400  -  4940  MHz  and 

5091  -  51 50  MHz  with  Spectrum. 

Regardless  of  mitigation,  loss  of 
capacity  cannot  be  mitigated  over 
longterm. 

1780 -1850  MHz 

L-Band  Telemetry 

F/EA-1 8G,  Aerostar,  ASVS,  SM-2,  RAM, 
SSRT,  Classified  UAV  (WSMR),  ARAV, 
X-47,  the  only  band  for  miss-distance 
indicators  used  to  score  missile  shots 

This  spectrum  may  be  auctioned 
over  the  next  1 0  years.  DOD  working 
towards  sharing  vice  reallocation. 

2200  -  2290  MHz 

S-Band  Telemetry 

AIM-9X,  AIM-1 20,  JAASM,  JDAM, 
WCMD,  JSOW,  SDB,  Aerostar,  ASVS, 
WSI,  6DOF,  MDA,  Patriot,  SM-2, 
ATACMS,  F-1 5,  F-1 6,  F-22,  F-35,T-38, 
B-1,  B-2,  B-52,  C-1 7,  Global  Hawk, 

X-51  Waverider 

Band  has  been  found  to  be  exceptionally 
vulnerable  to  emissions  from  Long  Term 
Evolution  wireless  broadband  towers 
operating  more  than  50  MHz  below  the 
band  edge.  Mitigation  is  being  worked. 

2360  -  2390  MHz 

Upper  S-Band 
Telemetry 

F-1 8E/400,  E2-D,  P-8A,  Exdrone,  Silver 
Fox,  THAAD,  F-1 6,  F-22,  B-1 ,  B-2,  B-52, 
C-1 7,  Global  Hawk 

•  Issue:  Wireless  communications  in 
2345-2360  can  interfere  with  operations 
in  this  band. 

•  Potential  Mitigation:  Pending. 

Working  with  industry  to  try  to  solve 
interference  problems. 

2390  -  2395  MHz 

Upper  S-Band 
Telemetry 

F-1 8E/400,  E2-D,  P-8A,  Exdrone,  Silver 
Fox,  THAAD,  F-1 6,  F-22,  B-1 ,  B-2,  B-52, 
C-1 7,  Global  Hawk 

Shared  for  additional  Upper  S-Band 
coverage. 

2700  -  2900  MHz 

Range  surveillance 
radar 

Critical  to  almost  all  open-air  tests 

4400 -4940  MHz 

Range  Telemetry 

F-1 5SA,  F-1 5  (pending),  fixed 
point-to-point  microwave,  tactical 
radio,  UAV,  threat  simulators 

Band  is  just  now  coming  into  use. 

5091  -5150 

MHz  (Region  2: 
5091  -  6700  MHz) 

Range  Telemetry 

F-15SA 

Shared  with  Federal  Aviation 
Administration.  Band  is  just  now 
coming  into  use. 
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Joint  Test  and  Evaluation  (JT&E) 


The  primary  objective  of  the  Joint  Test  and  Evaluation 
(JT&E)  Program  is  to  provide  solutions  rapidly  to  operational 
deficiencies  identified  by  the  joint  military  community.  The 
program  achieves  this  objective  by  developing  new  tactics, 
techniques,  and  procedures  (TTP)  and  rigorously  measuring 
the  extent  to  which  their  use  improves  operational  outcomes. 
JT&E  projects  may  develop  products  that  have  implications 
beyond  TTP.  Sponsoring  organizations  submit  these  products 
to  the  appropriate  Service  or  Combatant  Command  as  doctrine 
change  requests.  Products  from  JT&E  projects  have  been 
incorporated  into  joint  and  multi-Service  documents  through 
the  Joint  Requirements  Oversight  Council  process  and  through 
coordination  with  the  Air,  Land,  Sea  Application  Center.  The 
JT&E  Program  also  develops  operational  testing  methods  that 
have  joint  application.  The  program  is  complementary  to,  but  not 
part  of,  the  acquisition  process. 

The  JT&E  Program  has  two  test  methods  available  for 
customers:  the  traditional  Joint  Test  and  the  Quick  Reaction  Test 
(QRT).  Additionally,  a  Special  Project  is  available  for  command 
directed  or  customer  funded  test  projects. 

The  traditional  Joint  Test  is,  on  average,  a  two-year  project, 
preceded  by  a  six-month  Joint  Feasibility  Study.  A  Joint 
Test  involves  an  in-depth,  methodical  test  and  evaluation  of 
issues  and  seeks  to  identify  their  solutions.  DOT&E  funds  the 
sponsor  led  test  team,  which  provides  the  customer  periodic 
feedback  and  useable,  interim  test  products.  The  JT&E  Program 
charters  two  new  Joint  Tests  annually.  The  JT&E  Program 
managed  seven  Joint  Tests  in  FY15  that  focused  on  the  needs  of 
operational  forces.  Projects  annotated  with  an  asterisk  (*)  were 
completed  in  FY15: 

•  Four  Pillars  of  Integrated  Air  and  Missile  Defense  (4-PI) 

•  Joint  Base  Architecture  for  Secure  Industrial  Control  Systems 
(J-BASICS) 

•  Joint  Counter  Low,  Slow,  Small  Unmanned  Aircraft  Systems 
(JCLU)* 

•  Joint-Fiber  Laser  Mission  Engagement  (J-FLaME) 

•  Joint  Pre-/Post-Attack  Operations  Supporting  Survivability 
And  Endurability  (J-POSSE) 

•  Joint  Tactical  Air  Picture  (JTAP) 

•  Unmanned  Aircraft  Systems  -  Airspace  Integration 
(UAS-AI)* 

QRTs  are  intended  to  solve  urgent  issues  in  less  than  a  year.  The 
program  managed  25  QRTs  in  FY15: 

•  Command  and  Control  of  Ballistic  Missile  Defense 
(C2BMD)* 

•  Cyber  Agility  and  Defensive  Maneuver  (CAADM)* 

•  Civil  Military  Engagement  Development  Joint 
Targeting/Non-Lethal  (CMED-JT/NL) 


•  Cyber  Threat  Information  Exchange  (CTIX)* 

•  Homeland  Underwater  Port  Assessment  Plan  (HUPAP) 

•  Joint  Assessment  Doctrine  Evaluation  (JADE)* 

•  Joint  Automated  Net-Centric  Satellite  Communications 
Electromagnetic  Interference  Resolution  (J-ANSER)* 

•  Joint  BiologicaPRadiological  Mortuary  Affairs  Contaminated 
Remains  Mitigation  Site  (JBRM) 

•  Joint-Cyber  Synchronization  into  Air  Tasking  Order  (J-CAT) 

•  Joint  Cyber  Integration  of  DOD  Information  Network 
Operations  (J-CID) 

•  Joint  Cyberspace  Intelligence,  Surveillance,  and 
Reconnaissance  (JCISR)* 

•  J oint  Decision  Support  -  Air  (JDeS-A)* 

•  Joint  Homeland  Mining  Prevention  and  Response  (JHMPR)* 

•  Joint  Integrated  Air  and  Ground  Situational  Awareness  (JIAG 
SA)* 

•  Joint  Intelligence  Surveillance  and  Reconnaissance  in  a 
Contested  Area  (JICA) 

•  Joint  Integrated  Standoff  Weapons  Employment  (JISOWE)* 

•  Joint  Laser  Anti-Satellite  Mitigation  Mission  Planning 
(J-LAMMP) 

•  Joint  National  Capital  Region  Air  Surveillance  Concept  of 
Operations  (CONOPS)  -  Accelerated  (JNASC-A)* 

•  Joint  Personnel  Recovery  Information  Digital  Exchange 
(J-PRIDE) 

•  Joint  Precise  Timing  (JPT)* 

•  Joint  Sniper  Performance  Improvement  Methodology 
(JSniPIM) 

•  Joint  Unmanned  Aerial  Vehicle  Swarming  Integration  (JUSI) 

•  Mortuary  Affairs  Contaminated  Remains  Mitigation  Site 
(MACRMS)* 

•  Theater  Joint  Land  Forces  Component  Commander  Common 
Operational  Picture  (T-COP) 

•  Joint  Target  Development:  Target  System  Analysis  Standards 
and  Procedures  (T-SaP) 

As  directed  by  DOT&E,  the  program  executes  Special  Projects 
that  address  DOD-wide  problems.  Special  Projects  generally 
address  emergent  issues  that  are  not  addressed  by  any  other  DOD 
agency,  but  that  need  a  rigorously  tested  solution.  The  program 
managed  three  Special  Projects  in  FY15: 

•  Joint  and  Community  Attributes-Based  Access  Control 
Authorization  for  Transportation  Services  (J-CAATS) 

•  Joint  National  Capital  Region  Enhanced  Surveillance  Tactics, 
Techniques,  and  Procedures  (J-NEST) 

•  Joint  Personnel  Recovery  Collaboration  and  Planning 
(JPRCaP)* 
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JOINTTESTS 


FOUR  PILLARS  OF  INTEGRATED  AIR  AND  MISSILE  DEFENSE 
(4-PI) 

Sponsor/Start  Date:  U.S.  European  Command,  U.  S.  Army 
Space  and  Missile  Defense  Command,  and  U.  S.  Air  Forces 
Europe-Air  Forces  Africa/August  2014 

Purpose:  To  develop  TTP  that  enable  sharing  of  existing  sensor 
data  to  enhance  the  concurrent  execution  of  integrated  air  and 
missile  defense  (IAMD)  active  defenses,  passive  defenses, 
attack  operations,  and  battle  management  command,  control, 
communications,  and  intelligence  in  response  to  ballistic  missile 
attacks  across  Combatant  Command  areas  of  responsibility 
(AOR)  in  a  coalition. 

Products/Benefits: 

•  TTP  that  share  data  to  support  concurrent  offensive  and 
defensive  counter-air  operations  in  order  to  better  defend 
against,  and  mitigate  the  effects  of,  a  ballistic  missile  attack 
across  AOR  boundaries  between  U.S.  European  Command, 
U.S.  Central  Command,  and  NATO. 

-  Utilize  missile  launch  point  of  origin  data  derived  from 
Overhead  Persistent  Infrared  systems  to  initiate  joint 
targeting  cycles  and  coordinate  targeting  priorities  across 
AORs 

-  Leverage  cross-AOR  sensor  data  sharing  to  provide  earlier 
warning;  enhance  ballistic  missile  radar  coverage,  threat 
detection,  track  management,  and  missile  engagement 
procedures;  and  system  redundancy 

-  Utilize  existing  radar  data  to  provide  refined  ballistic 
missile  impact  point  predictions  that  will  enhance  the 
effectiveness  of  early  warning  and  actionable  consequence 
management 

•  Standardizes  battle  management  command,  control, 
communications,  and  intelligence  capabilities  and  Global 
Command  and  Control  System  -  Joint  configurations 

to  maximize  efficiencies,  support  command  and  control 
collaboration,  and  enable  sharing  of  IAMD  sensor  data 

•  Supports  development  of  an  enhanced  civil-military  passive 
defense/missile  warning  process  for  NATO  nations,  extensible 
to  other  Combatant  Command  Shared  Early  Warning  partners 

•  Delivers  a  leave  behind  exercise  framework  for  cross-AOR 
IAMD  exercises 

•  Supports  the  National  Geospatial  Intelligence  Agency 
in  implementing  a  Chairman  of  the  Joint  Chiefs  of  Staff 
instruction-directed  common  area  and  point  reference  system 
across  Combatant  Commands — the  Global  Area  Reference 
System  and  the  Military  Grid  Reference  System 

JOINT-BASE  ARCHITECTURE  FOR  SECURE  INDUSTRIAL 
CONTROL  SYSTEMS  (J-BASICS) 

Sponsor/Start  Date:  U.S.  Cyber  Command 
(USCYBERCOMj/February  2014 

Purpose:  To  develop,  test,  and  evaluate  Advanced  Cyber 
Industrial  Control  System  (ICS),  or  (ACI),  TTP  to  improve  the 


ability  of  ICS  network  managers  to  detect,  mitigate,  and  recover 
from  nation-state  cyber  attacks. 

Products/Benefits: 

•  ACI  TTP  and  related  ICS  network  manager  training  packages 
will  provide  the  following  capabilities: 

-  Resiliency  (fight-through  capability)  to  DOD  ICS  networks 
and  immediate  supporting  IT  infrastructures 

-  Advanced  means,  in  the  form  of  TTP,  for  ICS  network 
managers  to:  detect  nation-state  presence  in  DOD  ICS 
networks;  mitigate  damage  to  underlying  processes 
supported  by  the  ICS  in  the  event  of  a  cyber  attack;  and 
quickly  recover  the  ICS  network  to  a  fully  mission-capable 
condition 

-  Increased  Commander  confidence  resulting  from  the  ability 
of  ICS  managers  to  accurately  detect  active  nation-state 
attacks  and  execute  defensive  measures  in  ICS  networks, 
ensuring  mission  readiness  of  ICS-dependent  activities 

-  Policy  and  implementation  guidance  recommendations  for 
ICS  network  security  to  Commander,  USCYBERCOM 
and  the  Assistant  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (Energy,  Installations  and 
Environment) 

JOINT  COUNTER  LOW,  SLOW,  SMALL  UNMANNED  AIRCRAFT 
SYSTEMS  (JCLU) 

(CLOSED  APRIL  201 5) 

Sponsor/Start  Date:  Air  Force/ August  2012 

Purpose:  To  develop,  test,  and  evaluate  IAMD  operator  TTP  to 
increase  operators’  ability  to  detect,  track,  and  identify  adversary 
low,  slow,  and  small  unmanned  aircraft  systems  (UAS)  and 
provide  timely  notification  to  the  Area  Air  Defense  Commander. 

Products/Benefits: 

•  TTP  that  increased  the  operators’  ability  to  detect,  track,  and 
identify  this  UAS  threat  category. 

•  Integrated  information  from  National  Technical  Means  into 
a  tactical  datalink  to  support  situational  awareness  and  target 
identification 

•  Developed  operational  architecture  and  organizational 
relationships  that  will  increase  the  cross-sharing  of  tactical 
information  to  increase  the  operators’  ability  to  execute  the 
joint  engagement  sequence 

JOINT  FIBER  LASER  MISSION  ENGAGEMENT  (J-FLAME) 

Sponsor/Start  Date:  Naval  Surface  Warfare  Center,  Dahlgren 
Division/August  2014 

Purpose:  To  develop  TTP  to  integrate  emerging  directed-energy 
laser  (DEL)  capabilities  to  conduct  joint  fires  and  force  protection 
missions. 

Products/Benefits: 

•  The  DEL  Operations  in  the  Joint  Battlespace  TTP  will: 
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-  Leverage  capabilities  of  emerging  DEL  to  conduct  joint 
fires  and  air  defense  missions 

-  Integrate  DEL  systems  into  joint  fires  planning  and 
execution,  focusing  on  coordinating  measures  needed  for 
deconfliction,  integration,  synchronization,  and  safety  of 
these  DEL  systems  in  a  complex  and  congested  battlespace 

-  Address  the  force  protection  mission  against  asymmetric 
threats  (UAS  and  small  boats),  focusing  on  unique  aspects 
of  DEL  that  impact  the  joint  battlespace  (for  example,  new 
or  different  coordinating  measures)  that  personnel  at  both 
operational  and  tactical  levels  need  to  consider 

JOINT  PRE-/POST-ATTACK  OPERATIONS  SUPPORTING 
SURVIVABILITY  AND  ENDURABILITY  (J-POSSE) 

Sponsor/Start  Date:  U.S.  Strategic  Command 
(U S  STRAT COM)/February  2015 

Purpose:  To  develop,  test,  and  evaluate  TTP  to  provide  joint 
operators  the  ability  to  survive  an  electromagnetic  pulse  (EMP) 
event  in  order  to  ensure  continuous  mission  functionality. 

Produets/Benefits: 

•  Standardized  procedures  that  provide  overarching  guidance 
for  required  actions  before  and  after  an  EMP  event  in  order  to 
survive  it. 

-  Pre-event  actions  include  preparations  taken  during  routine 
operations,  as  well  as  emergency  actions  taken  once 
notified  of  an  imminent  EMP  event 

-  Post-event  actions  include  endurability  operations  to 
maintain  mission  functionality  for  a  specified  period  of 
time  following  an  EMP  event 

•  Results  inform  future  resourcing  decisions  regarding  physical 
enhancements 

•  Extensible  to  other  mission  systems  potentially  vulnerable  to 
EMP  effects  (e.g.  missile  defense,  space,  cyber) 

JOINTTACTICAL  AIR  PICTURE  (JTAP) 

Sponsor/Start  Date:  U.S.  Pacific  Command 
(USPACOM)/F ebruary  2014 

Purpose:  To  develop,  evaluate,  and  validate  TTP  to  improve  the 
joint  air  picture  and  engagement  opportunities,  which  decreases 
the  risk  of  preemptive  hostile  attack  and  fratricide. 


Products/Benefits: 

•  Link  1 6  implementation  procedures  that  reduce  radio 
frequency  network  loading  by  moving  participants  to  Internet 
protocol  architecture  resulting  in  a  greater  number  of  timeslots 
available  for  participants 

•  Multi-Service  Integrated  Air  and  Missile  Defense  TTP  that 
enhances  integrated  fire  control  between  ground  sensors  and 
air  shooters  for  defensive  counter  air  engagements  thereby 
increasing  the  number  of  available  tracks  containing  fire 
control  quality  data 

UNMANNED  AIRCRAFT  SYSTEMS  AIRSPACE  INTEGRATION 
(UAS-AI) 

(CLOSED  JULY  201 5) 

Sponsor/Start  Date:  North  American  Aerospace 
Defense  Command  (NORAD)-U.S.  Northern  Command 
(USNORTHCOM),  and  the  Army  Test  and  Evaluation 
Command/ August  2012 

Purpose:  Standardize  and  evaluate  procedures  to  safely, 
effectively,  and  efficiently  operate  UAS  in  the  National  Airspace 
System  (NAS). 

Products/Benefits: 

•  Increased  collaboration  between  the  DOD  Policy  Board  on 
Federal  Aviation,  Federal  Aviation  Administration,  National 
Aeronautics  and  Space  Administration,  and  Customs  and 
Border  Protection  on  UAS  NAS  integration  issues 

•  UAS  NAS  Flight  Operations  Standardized  Procedures  for 
operating  UAS  in  the  NAS  under  routine,  lost  command  link, 
lost  two-way  radio  communications,  and  lost  detect  and  avoid 
conditions 

•  UAS-specific  aeronautical  charting  guidelines  for  use  by  the 
National  Geospatial-Intelligence  Agency 

•  Chase  Aircraft  and  Airborne  Visual  Observer  TTP  for 
pre -mission  planning  and  briefings  that  lead  to  safe  UAS 
operations  utilizing  chase  aircraft  in  the  NAS 


QUICK  REACTION  TESTS 


COMMAND  AND  CONTROL  OF  BALLISTIC  MISSILE  DEFENSE 
(C2BMD) 

(CLOSED  JUNE  201 5) 

Sponsor/Start  Date:  USSTRATCOM,  Air  Force  Joint  Test 
Program  Office/February  2014 

Purpose:  To  develop  and  test  TTP  leveraging  current  Command 
and  Control,  Battle  Management  and  Communications  system 
capabilities  resident,  but  not  fully  utilized,  to  enhance  intra-  and 
inter-theater  joint  ballistic  missile  defense  (BMD)  operations 
planning  and  re -planning  efforts. 


Products/Benefits: 

•  Improved  BMD  coordination  among  the  Air  Operations 
Center,  Maritime  Operations  Center,  and  Army  Air  and  Missile 
Defense  Command  in  support  of  intra-  and  inter-theater  BMD 
operations 

•  Enhanced  ability  of  theaters  to  successfully  plan  and  employ 
limited  organic  BMD  assets 
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•  Improved  exploitation  of  capabilities  resident,  but 
underutilized,  in  fielded  Command  and  Control,  Battle 
Management  and  Communications  6.4  software 

CYBER  AGILITY  AND  DEFENSIVE  MANEUVER  (CAADM) 
(CLOSED  JANUARY  201 5) 

Sponsor/Start  Date:  USPACOM/August  2013 

Purpose:  To  develop  and  test  TTP  to  enhance  moving  target 
technologies  to  enable  cyber  agility  and  defensive  cyber 
maneuver  for  the  protection  of  selected  critical  information 
resources  and  command  and  control  capabilities  from  advanced 
threats.  Also,  to  provide  recommendations  for  amendments  of 
joint  doctrine  (principally  Joint  Publication  3-12,  Cyberspace 
Operations)  to  introduce  more  comprehensive  operational 
concepts  for  defensive  maneuver  in  cyberspace. 

Products/Benefits: 

•  Developed  TTP  and  recommended  changes  to  joint  doctrine  to 
provide  the  following: 

-  Assisted  Commanders  and  network  defenders  in 
overcoming  disadvantages  inherent  in  static  cyber  defenses 

-  Decreased  vulnerability  to  enemy  surveillance  of  and 
attacks  against  DOD  network  enclaves 

-  Enhanced  ability  to  rapidly  adapt  cyber  defenses  in  the  face 
of  changing  missions  and  threats 

-  Improved  capability  to  counter  and  observe  enemy  actions 
in  cyberspace 

-  Increased  wherewithal  to  shift  initiative  from  attackers  to 
network  defenders 

-  More  effective  application  of  technology  for  agile  defense 
of  key  terrain  in  cyberspace 

-  Developed  the  foundation  for  more  effective  joint  planning 
and  operations  for  defense  of  critical-enabled  capabilities 
and  information  resources 

CIVIL  MILITARY  ENGAGEMENT  DEVELOPMENT-JOINT 
TARGETING/NON-LETHAL  (CMED-JT/NL) 

Sponsor/Start  Date:  U.S.  Army  Civil  Affairs  &  Psychological 
Operations  Command  (Airbornej/February  2015 

Purpose:  To  develop,  test,  and  validate  civil-military 
engagement  development  (CMED)  TTP  to  improve  the  non-lethal 
aspects  of  the  joint  targeting  process.  To  increase  the  Combatant 
Command  staff’s  ability  to  integrate  civil  information  and 
analysis  products  into  the  joint  targeting  cycle  and  improve  basic, 
intermediate,  and  advanced  joint  target  folder  development, 
entity-level  development,  prioritization  (phase  two  of  the  joint 
targeting  process),  and  no  strike  and  restricted  target  lists. 

Products/Benefits: 

The  CMED-JT/NL-developed  TTP  will  provide  Commanders  the 
ability  to  integrate  civil  military  information  into  phase  two  of  the 
joint  targeting  process. 

CYBERTHREAT  INFORMATION  EXCHANGE  (CTIX) 

(CLOSED  SEPTEMBER  201 5) 

Sponsor/Start  Date:  USPACOM/June  2014 


Purpose:  To  develop  and  test  TTP  for  timely,  relevant  exchange 
of  cyber  threat  data  with  joint  and  coalition  cyber  defenders 
and  cyber  operations  centers  in  support  of  joint  and  combined 
operations  with  the  focus  on  developing  effective  formats  and 
processes  to  enable  automated  and  man-in-the-loop  cyber  threat 
information  exchanges. 

Products/Benefits:  A  CTIX-developed  TTP  to  exchange 
relevant  cyber  threat  data  between  joint  and  coalition  partners 
and  cyber  operations  centers  during  both  joint  and  combined 
operations. 

HOMELAND  UNDERWATER  PORT  ASSESSMENT  PLAN 
(HUPAP) 

Sponsor/Start  Date:  NORAD-USNORTHCOM/June  2015 

Purpose:  To  develop  and  assess  TTP  for  underwater  port 
assessments  to  include:  specific  details  about  the  roles  and 
responsibilities  of  the  stakeholders;  identify  available  local, 
state,  and  federal  force  multipliers;  provide  data  collection, 
compilation,  and  sharing  guidance;  and  identify  gaps  in  response 
considerations. 

Products/Benefits: 

•  TTP  that  provides  specific  details  of  conducting  an  underwater 
port  assessment 

•  A  supporting  implementation  plan  that  prescribes  all  aspects  of 
manning,  funding  support,  and  coordination  to  execute  these 
critical  assessments 

JOINT  ASSESSMENT  DOCTRINE  EVALUATION  (JADE) 

(CLOSED  JUNE  201 5) 

Sponsor/Start  Date:  U.S.  Central  Command/February  2014 

Purpose:  To  develop  and  improve  the  integration  of  Theater 
Campaign  and  Operation  Assessment  into  the  planning  and 
execution  phases  at  the  Joint  Force  Commander  level.  To 
document  and  inform  the  Joint  Doctrine  Note  being  developed  by 
the  Joint  Staff  J7,  including  specific  changes  to  Joint  Publication 
(JP)  3-0,  Joint  Operations,  and  JP  5-0,  Joint  Operation  Planning. 

Products/Benefits:  Findings  and  recommendations  regarding 
doctrine,  leadership  and  education,  and  training;  draft  comments 
and  text  for  inclusion  in  revisions  to  JP  3-0  and  JP  5-0;  including 
suggested  terminology  changes  associated  with  the  operation 
assessment  lexicon. 

JOINT  AUTOMATED  NET-CENTRIC  SATELLITE 
COMMUNICATIONS  ELECTROMAGNETIC  INTERFERENCE 
RESOLUTION  (J-ANSER) 

(CLOSED  NOVEMBER  201 4) 

Sponsor/Start  Date:  USSTRATCOM,  Air  Force  Joint  Test 
Program  Office/November  2013 

Purpose:  To  develop  satellite  communications  (SATCOM) 
electromagnetic  interference  TTP  leveraging  recently  fielded, 
net-centric  systems  to  immediately  detect,  characterize,  and  cue 
geolocation  assets. 
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Products/Benefits: 

•  Allowed  Commanders  and  operators  to  advance  operations 
in  a  SATCOM  denied  or  degraded  environment  by  visually 
displaying  SATCOM  lines  of  communication  health  and  status 

•  Improved  SATCOM  operators  responsiveness  to  SATCOM 
interference 

•  Enhanced  real-time  situational  awareness  by  reducing 
SATCOM  electromagnetic  interference  resolution  timelines 
from  hours  to  minutes 

•  Integrated  SATCOM  common  operating  picture  TTP  among 
the  SATCOM  operators 

JOINT  BIOLOGICAL/RADIOLOGICAL  MORTUARY  AFFAIRS 
CONTAMINATED  REMAINS  MITIGATION  SITE  (JBRM) 

Sponsor/Start  Date:  U.S.  Army  Quartermaster 
School/ June  2015 

Purpose:  To  develop  and  assess  TTP  for  the  safe  processing, 
identification,  and  preparation  for  evacuation  of  biologically 
or  radiologically  contaminated  human  remains.  To  improve 
the  Mortuary  Affairs  Contaminated  Remains  Mitigation  Site 
effectiveness  and  safety  for  operational  mission  requirements, 
including  hazard  mitigation,  preserving  forensic  evidence, 
establishing  chain  of  custody,  supporting  positive  identification 
processes,  and  preparing  remains  for  evacuation. 

Products/Benefits: 

•  Updates  to  Army  and  joint  doctrine,  with  primary  focus 
on  Army  Techniques  Publication  4-46.2,  Mortuary  Affairs 
Contaminated  Remains  Mitigation  Site  Operations,  as  related 
to  biological  or  radiological  contaminated  human  remains 

•  Verifiable  data  and  tools  to  the  mortuary  affairs  community 
for  the  purpose  of  improving  TTP  for  use  in  both 
USNORTHCOM  homeland  defense  missions  and  DOD’s 
worldwide  contingency  operations 

JOINT-CYBER  SYNCHRONIZATION  INTO  AIRTASKING  ORDER 
(J-CAT) 

Sponsor/Start  Date:  USPACOM/October  2014 

Purpose:  To  develop  and  test  TTP  for  Combatant  Commands  to 
direct  regionally  synchronized  and  globally  deconflicted  cyber 
fires.  Specifically,  TTP  will  integrate  offensive  cyberspace 
operations  into  a  Combatant  Command’s  air  tasking  order 
development  and  execution  processes  in  order  to  synchronize 
cyber  operations  with  other  joint  fires,  as  well  as  provide 
coordination  and  deconfliction  of  global  cyber  operations  with 
USCYBERCOM’s  cyberspace  tasking  order. 

Products/Benefits:  TTP  that  provide  the  best  practices  for 
synchronization  of  cyber  fires  into  the  joint  fires  process  for 
deconfliction  of  regional  cyber  fires  with  global  cyberspace 
operations. 


Purpose:  To  develop  a  concept  of  operations  (CONOPS)  and 
TTP  for  the  Combatant  Commands’  Joint  Cyber  Center  that  will 
enable  them  to  fully  integrate  the  organization,  authorities,  and 
capabilities  of  DOD  Information  Network  commands  in  support 
of  joint  theater  cyber  operations. 

Products/Benefits:  CONOPS  and  TTP  that  provide  best 
practices  for  the  support  of  regional  operations,  situational 
understanding,  and  decision  making  for  cyberspace  operations 
between  regional  DOD  Information  Network  commands  and 
regional  Joint  Cyber  Centers. 

JOINT  CYBERSPACE  INTELLIGENCE,  SURVEILLANCE,  AND 
RECONNAISSANCE  (JCISR) 

(CLOSED  JANUARY  201 5) 

Sponsor/Start  Date:  USPACOM/January  2014 

Purpose:  To  develop  and  evaluate  TTP  that  enable  a  Joint  Cyber 
Center  to  integrate  the  intelligence  community’s  (IC)  cyberspace 
Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  products 
into  joint  operation  planning,  joint  targeting,  and  operations. 

Products/Benefits: 

•  Established  processes  for  full  integration  of  the  IC  cyberspace 
ISR  products  into  planning,  targeting,  and  execution  of 
offensive  cyber  operations  by  Combatant  Commander’s  Joint 
Cyber  Center 

•  Provided  a  framework  for  the  IC  to  communicate  with 
newly-formed  Combatant  Command  Joint  Cyber  Center 

•  Developed  a  doctrine,  organization,  training,  materiel, 
leadership  and  education,  personnel,  facilities  and  policy 
change  request  on  factors  that  impede  IC  support  for  offensive 
cyber  operations 

•  Validated  TTP  through  an  assessment  of  developed  processes 
across  Combatant  Commands 

JOINT  DECISION  SUPPORT-  AIR  (JDES-A) 

(CLOSED  NOVEMBER  201 4) 

Sponsor/Start  Date:  NORAD/November  2013 

Purpose:  To  develop  and  test  TTP  for  use  by  operators  of  the 
Air/Event  Information  Sharing  Service  (A/EISS)  that  prevent 
incomplete  or  inaccurate  air  event  data  from  being  provided  to 
senior  decision  makers.  To  enhance  situational  awareness  and 
enable  the  Commander,  NORAD  and  USNORTHCOM,  Civil 
Aircraft  Engagement  Authorities,  Canadian  Recommending 
Authorities,  and  all  participating  joint,  interagency, 
intergovernmental,  and  multinational  air  defense  and  security 
mission  partners  to  make  timely  decisions  during  air  events  over 
North  America. 

Products/Benefits:  Enhanced  A/EISS  TTP  and  training  and 
evaluation  products  to  deliver  air  domain  situational  awareness 
and  decision  support  in  support  of  Operation  Noble  Eagle. 


JOINT  CYBER  INTEGRATION  OF  DOD  INFORMATION 
NETWORK  OPERATIONS  (J-CID) 

Sponsor/Start  Date:  USPACOM/June  2015 
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JOINT  HOMELAND  MINING  PREVENTION  AND  RESPONSE 
(JHMPR) 

(CLOSED  NOVEMBER  2014) 

Sponsor/Start  Date:  NORAD-USNORTHCOM/August  2013 

Purpose:  To  develop  a  cross-departmental,  interagency  CONOPS 
to  enable  a  mine  countermeasures  response  to  a  mine  incident  in 
a  port  in  the  United  States. 

Products/Benefits:  CONOPS  for  the  rapid,  accurate,  and 
standardized  information  exchange  between  the  Department  of 
Homeland  Security  mission  of  mine  detection  and  prevention  and 
the  DOD’s  mine  countermeasures  response. 

JOINT  INTEGRATED  AIR  AND  GROUND  SITUATIONAL 
AWARENESS  (JIAG  SA) 

(CLOSED  SEPTEMBER  201 5) 

Sponsor/Start  Date:  Joint  Staff  J6/June  2014 

Purpose:  To  provide  TTP  to  employ  the  Tactical  Radio 
Application  eXention  (TRAX)  to  integrate  multiple 
tactical  datalinks  through  a  joint  integrated  air  and  ground 
common  operational  picture.  This  TTP  increases  the  tactical 
decision-makers’  situational  awareness,  shortens  the  kill  chain, 
reduces  the  potential  of  fratricide,  and  facilitates  airspace 
de-confliction  and  rapid  target  correlation  for  digitally-aided 
operations. 

Products/Benefits:  A  joint  special  operations  forces  TRAX  TTP 
that  describes  set-up  and  installation,  platform  specific  integration 
considerations,  and  tactics  for  TRAX  integration  in  selected 
forcible  entry  operation  scenarios. 

JOINT  INTELLIGENCE  SURVEILLANCE  AND 
RECONNAISSANCE  [ISR]  IN  A  CONTESTED  AREA  (JICA) 

Sponsor/Start  Date:  25th  Air  Force/October  2014 

Purpose:  To  develop,  test,  and  validate  TTP  to  improve 
information  flow  from  national  ISR  capabilities  to 
operational  and  tactical-level  customers  in  an  anti-access/ area 
denial  environment.  To  provide  criteria  to  define  an  ISR 
denial/degradation  “trigger”  to  begin  assembling  data  collected 
by  national  assets  against  pre-identified  targets  and  procedures  for 
the  Air  and  Space  Operations  Center  ISR  Division  personnel  to 
request  ISR  support  from  national  ISR  capabilities  to  supplement 
or  replace  theater  and  tactical  ISR  assets  that  cannot  be  utilized 
due  to  threat  vulnerability. 

Products/Benefits: 

•  TTP  for  Air  and  Space  Operations  Center  or  Maritime 
Operations  Center  ISR  Division  personnel  to  effectively 
obtain  access  to  national  intelligence  information  in  an 
anti-access/ area  denial  environment 

•  Allow  operational  and  tactical  warfighters  a  better  chance 
of  survivability  and  mission  success  by  ensuring  the  most 
accurate  intelligence  is  provided  to  them,  even  when  theater 
and  tactical  ISR  assets  are  degraded  or  denied 


JOINT  INTEGRATED  STANDOFF  WEAPONS  EMPLOYMENT 
(JISOWE) 

(CLOSED  JULY  201 5) 

Sponsor/Start  Date:  USPACOM/February  2014 

Purpose:  To  develop,  test,  and  evaluate  TTP  to  employ  standoff 
weapon  systems  that  include  the  Joint  Air-to-Surface  Standoff 
Missile,  Tomahawk  Land  Attack  Missile,  Conventional  Air 
Launched  Cruise  Missile,  Miniature  Air  Launched  Decoy,  and  the 
EA-18G  aircraft. 

Products/Benefits: 

•  Operational-level  planning  TTP  that  integrated  standoff 
weapon  systems 

•  A  report  of  the  capabilities  and  limitations  of  existing  standoff 
weapons,  decoys,  and  airborne  jammer  systems 

•  A  report  on  which  modeling  and  simulation  tools  provided  the 
best  operational  planning  capability 

JOINT  LASER  ANTI-SATELLITE  MITIGATION  MISSION 
PLANNING  (J-LAMMP) 

Sponsor/Start  Date:  U.S.  Air  Force  Warfare  Center/June  2014 

Purpose:  To  develop  and  evaluate  TTP  to  mitigate  anti-satellite 
threats  from  stationary,  ground-based,  low-power  lasers  targeting 
low  Earth  orbit  satellites  with  an  optical  payload. 

Produets/Benefits: 

•  TTP  that  incorporate  payload  susceptibility  information  and 
Commander’s  risk  acceptance  level  into  mission  planning  at 
both  the  operational  and  tactical  levels  of  space  operations 

•  TTP  will  provide  a  standard,  methodical  process  to  respond 
to  laser  anti-satellite  threats  by  assisting  operational-level 
users  to  identify  risks,  determine  the  decision  authority’s 
risk  acceptance  level,  notify  affected  tactical  users,  and 
assisting  tactical  users  to  develop  courses  of  action  to  mitigate 
risk  while  maintaining  support  and  communication  to  the 
warfighter  and  feedback  to  the  operational  level 

JOINT  NATIONAL  CAPITOL  REGION  AIR  SURVEILLANCE 
CONCEPT  OF  OPERATIONS  -  ADVANCED  (JNASC-A) 

(CLOSED  OCTOBER  201 4) 

Sponsor/Start  Date:  NORAD-USNORTHCOM/August  2013 

Purpose:  To  develop  and  test  CONOPS  and  TTP  to  integrate  the 
Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor 
System  (JLENS)  into  NORAD’s  Battle  Control  System  -  Fixed 
(BCS-F).  To  develop  procedures  to  enable  JLENS  to  detect, 
track,  and  characterize  items  of  interest  within  the  JLENS  field  of 
view  supporting  the  air  surveillance  of  the  national  capital  region 
and  surrounding  areas  to  positively  identify  contacts  and  increase 
the  time  available  to  take  action  to  counter  air  and  missile  threats. 

Products/Benefits:  TTP  for  JLENS  allowed  Eastern  and  Western 
Air  Defense  Sector  operators  to  efficiently  and  effectively  exploit 
available  detection  and  identification  capabilities  to  improve  the 
NORAD  homeland  defense  mission  within  the  national  capital 
region. 
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JOINT  PERSONNEL  RECOVERY  INFORMATION  DIGITAL 
EXCHANGE  (J-PRIDE) 

Sponsor/Start  Date:  Joint  Staff  J7/June  2015 

Purpose:  To  develop  and  test  TTP  to  pass  critical  information 
across  existing  hybrid  networks  between  isolated  personnel, 
recovery  forces,  and  command  and  control  nodes  during  joint 
personnel  recovery  (PR)  mission  execution. 

Products/Benefits: 

•  TTP  to  provide  efficient  and  effective  use  of  digitally 
generated  mission  critical  information  across  operational 
and  tactical  PR  nodes  to  enhance  mission  effectiveness  and 
increase  survivability 

•  To  provide  guidance  to  achieve  complete,  accurate,  timely,  and 
persistent  joint  PR  capabilities 

JOINT  PRECISE  TIMING  (JPT) 

(CLOSED  JANUARY  201 5) 

Sponsor/Start  Date:  Office  of  the  Chief  of  Naval 
Operations/ August  2013 

Purpose:  To  develop  and  test  TTP  to  provide  overarching 
guidance  and  best  practices  for  the  standardization  and  operation 
of  DOD  Precise  Time  and  Time  Interval  systems. 

Products/Benefits: 

•  Provided  guidance  for  the  standardization  and  operation  of 
DOD  Precise  Time  and  Time  Interval  distribution  systems 

•  Formalized  and  documented  best  practices  and  guidelines  to 
improve  the  reliability,  redundancy,  and  assurance  of  DOD 
systems 

•  Enabled  the  joint  force  to  achieve  and  sustain  accurate, 
synchronized  time  of  day  and  frequency  worldwide  to  support 
joint  operations 

JOINT  SNIPER  PERFORMANCE  IMPROVEMENT 
METHODOLOGY  (JSNIPIM) 

Sponsor/Start  Date:  U.S.  Marine  Corps  Weapons  Training 
Battalion/October  2014 

Purpose:  To  develop  and  test  TTP  and  training  methodologies 
to  confidently  employ  sniper  teams  and  focus  on  their  ability  to 
identify,  range,  lead,  and  engage  human  motion-type  moving 
targets  at  distances  of  300  to  1,000  meters  at  speeds  of  up  to 
10  miles  per  hour. 

Products/Benefits: 

•  A  sniper-carried  memory  aid  and  a  training  support  package 
that  includes  learning  objectives,  period  of  instruction 
materials  (including  a  training  video),  an  instructor  guide,  and 
a  student  handout 

•  Enable  instructors  to  teach,  test,  and  qualify  students  on 
engaging  moving  targets  at  distances  of  300  to  1,000  meters  at 
speeds  of  up  to  10  miles  per  hour 

•  Update  curriculums  for  all  DOD  sniper  schools 


Purpose:  To  develop  and  test  a  TTP  and  concept  of  employment 
to  plan  and  execute  integrated  attacks  using  unmanned  aerial 
vehicle  swarms  carrying  electronic  warfare  payloads  against 
advanced  air  defense  systems. 

Products/Benefits: 

•  A  concept  of  employment  and  TTP  to  plan  and  execute  joint 
integrated  standoff  attacks  against  a  modern  air  defense  system 

•  Identified  capabilities  and  limitations  of  existing  planning  and 
modeling  and  simulation  tools  for  this  mission  area 

MORTUARY  AFFAIRS  CONTAMINATED  REMAINS 
MITIGATION  SITE  (MACRMS) 

(CLOSED  JUNE  201 5) 

Sponsor/Start  Date:  U.S.  Army  Quartermaster 
School/February  2014 

Purpose:  To  assess  and  validate  TTP  for  processing 
contaminated  human  remains  and  their  personal  effects  resulting 
from  a  chemical,  biological,  radiological,  or  nuclear  incident. 

Products/Benefits: 

•  Updated  Chapter  4,  Processing  of  Chemical  Contaminated 
Human  Remains,  in  the  U.S.  Army  Techniques  Publication 
(ATP)  4-46.2,  Mortuary  Affairs  Contaminated  Remains 
Mitigation  Site 

•  Provided  changes  to  ATP  4-46.2  based  on  data  and 
observations 

THEATER  JOINT  LAND  FORCES  COMPONENT  COMMANDER 
COMMON  OPERATIONAL  PICTURE  (T-COP) 

Sponsor/Start  Date:  USPACOM/February  2015 

Purpose:  To  develop,  test,  and  validate  TTP  for  USPACOM’s 
land  components  and  implement  a  standard  operating  procedure 
for  Combatant  Command-level  operations  that  establish  a  T-COP 
across  the  Services. 

Produets/Benefits:  Enhanced  command  and  control  capability 
to  conduct  timely  analysis  and  recommendations  to  support  crisis 
and  contingency  responses. 

JOINTTARGET  DEVELOPMENT:  TARGET  SYSTEM  ANALYSIS 
STANDARDS  AND  PROCEDURES  (T-SAP) 

Sponsor/Start  Date:  Joint  Staff  J2/February  2015 

Purpose:  To  develop,  test,  and  validate  TTP  for  targeteers  and 
intelligence  analysts  to  conduct  target  system  analysis  in  support 
of  target  development  for  joint  force  operations. 

Products/Benefits: 

•  Target  system  analysis  TTP  to  support  joint  force  planning 
and  update  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
3370.01,  Target  Development  Standards 

•  A  findings,  conclusions,  and  recommendations  memorandum 
on  applicable  doctrine  change  recommendations  that  will  be 
transitioned  to  the  Joint  Staff  J2 


JOINT  UNMANNED  AERIAL  VEHICLE  SWARMING 
INTEGRATION  (JUSI) 

Sponsor/Start  Date:  USPACOM/February  2015 
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SPECIAL  PROJECTS 


JOINT  AND  COMMUNITY  ATTRIBUTES-BASED  ACCESS 
CONTROL  AUTHORIZATION  FOR  TRANSPORTATION 
SERVICES  (J-CAATS) 

Sponsor/Start  Date:  U.S.  Transportation 
Command/February  2015 

Purpose:  To  develop  and  validate  TTP  and  CONOPS  for 
providing  secure,  yet  timely  and  appropriate,  data  access  for 
DOD  users  using  an  attributes-based  access  control  approach. 

Products/Benefits:  A  process  for  utilizing  user  attributes  that 
eliminates  the  need  to  establish  a  new,  unique  account  often 
requiring  additional  usernames  and  passwords  when  a  new  user 
seeks  access  to  an  information  system. 

JOINT  NATIONAL  CAPITAL  REGION  ENHANCED 
SURVEILLANCE  TACTICS,  TECHNIQUES,  AND  PROCEDURES 
(J-NEST) 

Sponsor/Start  Date:  NORAD/October  2014 

Purpose:  To  develop,  test,  and  validate  TTP  to  incorporate 
emerging  sensor  capabilities  into  the  NORAD  and 
USNORTHCOM  family  of  systems  to  support  the  air  defense 
mission. 

Products/Benefits: 

•  Provide  TTP  that  enable  tactical,  operational,  and  strategic 
command  and  control  nodes  to  more  fully  employ  the 
expanded  detection,  improved  identification,  and  enhanced 
engagement  of  cruise  missile  threats  to  the  national  capital 
region 

•  Seamlessly  integrate  these  critical  capabilities  and  leverage  the 
full  benefit  of  an  advanced  family  of  sensors  into  the  “no  fail” 
mission  of  defending  the  national  capital  region 


•  Codify  processes  and  standardize  TTP  utilizing  advanced 
equipment  capabilities  to  execute  an  effective  joint 
engagement  sequence  for  cruise  missile  defense 

JOINT  PERSONNEL  RECOVERY  COLLABORATION  AND 
PLANNING  (JPRCAP) 

(CLOSED  JANUARY  201 5) 

Sponsor/Start  Date:  Joint  Personnel  Recovery 
Agency/ January  2013 

Purpose:  To  develop,  test,  and  evaluate  procedures  that  will 
formalize  planning,  crisis  response,  and  information  sharing 
between  the  geographic  Combatant  Commands  (GCC)  and 
defense  attaches  (DATT)  prior  to  and  during  personnel  recovery 
responses  where  a  State  Department  chief  of  mission,  generally 
the  ambassador,  and  not  a  DOD  official,  is  the  lead  U.S. 
Government  authority  for  activity  in  a  country. 

Products/Benefits: 

•  Formal  personnel  recovery  planning  and  training  protocols  that 
informed  GCCs  and  DATTs  in  U.S.  embassies  on  how  to  build 
and  implement  country-specific  personnel  recovery  plans  to 
develop  DOD  support  options  tailored  for  the  chief  of  mission 

•  A  handbook  that  provided  DATTs  a  reference  for  the  who, 
what,  and  why  of  responsibilities  for  personnel  recovery  when 
DOD  supports  a  chief  of  mission-led  personnel  recovery 
response 

•  A  web-based  program,  Automated  Development  Tool,  that 
took  roles,  responsibilities,  and  activities,  and  used  GCC  and 
DATT  inputs  to  produce  a  country-specific  DOD  personnel 
recovery  supplement 
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Center  for  Countermeasures  (CCM) 


The  Center  for  Countermeasures  (the  Center)  is  a  joint  activity 
that  directs,  coordinates,  supports,  and  conducts  independent 
countermeasure/counter-countermeasure  (CM/CCM)  test 
and  evaluation  (T&E)  activities  of  U.S.  and  foreign  weapons 
systems,  subsystems,  sensors,  and  related  components.  The 
Center  accomplishes  this  work  in  support  of  the  Director, 
Operational  Test  and  Evaluation  (DOT&E),  Deputy  Assistant 
Secretary  of  Defense  for  Developmental  Test  and  Evaluation 
((DASD(DT&E)),  weapon  systems  developers,  and  the  Services. 
The  Center’s  testing  and  analyses  directly  support  evaluations 
of  the  operational  effectiveness  and  suitability  of  CM/CCM 
systems. 

Specifically,  the  Center: 

•  Determines  performance  and  limitations  of  missile  warning 
and  aircraft  survivability  equipment  (ASE)  used  on 
rotary-wing  and  fixed-wing  aircraft 

•  Determines  effectiveness  of  precision  guided  weapon  (PGW) 
systems  and  subsystems  when  operating  in  an  environment 
degraded  by  CMs 

•  Develops  and  evaluates  CM/CCM  techniques  and  devices 

•  Operates  unique  test  equipment  that  supports  testing  across  the 
DOD 

•  Provides  analyses  and  recommendations  on  CM/CCM 
effectiveness  to  Service  Program  Offices,  DOT&E, 
DASD(DT&E),  and  the  Services 

•  Supports  Service  member  exercises,  training,  and 
pre -deployment  activities 

During  FY15,  the  Center  completed  over  35  T&E  activities. 

The  Center’s  support  to  these  activities  resulted  in  analysis 
and  reporting  on  more  than  27  DOD  electro-optical  systems  or 
subsystems,  with  special  emphasis  on  rotary-wing  survivability. 


The  Center  participated  in  operational/developmental 
tests  for  rotary-  and  fixed-wing  ASE,  PGWs,  hostile  fire 
indicator  (HFI)  data  collection,  experimentation  tests,  and 
pre -deployment/ exercise  support  using  CM/CCM.  To  best 
represent  the  level  of  effort  resourced  to  support  T&E,  the  Center 
tracks  funding  expended  in  each  test  area: 

•  Approximately  51  percent  of  the  Center’s  efforts  were  spent 
on  ASE  testing,  with  the  majority  of  these  efforts  in  support  of 
rotary-wing  aircraft 

•  About  22  percent  of  the  Center’s  efforts  were  spent  on  PGW, 
foreign  system,  and  other  types  of  field  testing  not  related  to 
ASE 

•  Approximately  7  percent  of  the  Center’s  efforts  were 
dedicated  to  overseas  contingency  operations  support, 
with  emphasis  on  CM-based  pre -deployment  training  for 
rotary-wing  units 

•  Approximately  17  percent  of  the  Center’s  efforts  were  spent 
on  internal  programs  to  improve  test  capabilities  and  to 
develop  test  methodologies  for  new  types  of  T&E  activities 

-  The  Center  continued  to  improve,  develop,  and 
validate  multiple  test  tools  for  evaluating  ASE  infrared 
countermeasure  (IRCM)  systems  and  F1FI  systems 

-  In  addition,  the  Center  is  improving  its  electronic  warfare 
capability  by  developing  and  validating  the  high-power 
Portable  Range  Threat  Simulator  (PRTS)  that  will  provide 
a  more  comprehensive,  integrated  ASE  T&E  environment 

•  The  Center  dedicated  about  3  percent  of  its  efforts  to 
providing  subject  matter  expertise  to  numerous  working 
groups  (WGs)  and  task  forces 

The  activities  conducted  by  the  Center  during  the  past  year  are 
detailed  in  the  subsections  that  follow. 


ASE  AND  HFI  ACTIVITIES 


RESEARCH  AND  DEVELOPMENT  ACTIVITY 

Foreign:  Trial  OXIDIZER  2 

•  Sponsor:  The  Center/Joint  Countermeasures  Test  and 
Evaluation  (JCMT&E)  WG 

•  Activity:  The  Center  collected  radiometric  signature  data  on 
weapon  firings  at  the  Mount  Bundey  Training  Area,  Northern 
Territory,  Australia.  Participation  was  under  the  provisions 
of  the  bilateral  U.S. /Australia  ASE  Cooperative  Test  and 
Evaluation  Project  Arrangement. 

•  Benefit:  Data  collected  in  the  hot,  humid  environment  during 
OXIDIZER  2  will  be  used  to  compare  to  data  collected  in  less 
humid  environments  to  better  understand  sensor  performance 
in  high  humidity  environments  and  to  scale  models  for  the 
effects  of  humidity. 


Foreign:  Infrared  Threat  Warning  System  Technical 

Demonstration  Program 

•  Sponsor:  The  Center/JCMT&E  WG 

•  Activity:  The  Center  provided  subject  matter  expertise  and 
assisted  with  planning  the  collection  of  Time-Space-Position 
Information  (TSPI)  before  the  United  Kingdom  (UK)  Defence 
Science  and  Technology  Laboratory  conducted  hostile  fire 
indication  testing  at  the  [UK’s]  Ministry  of  Defence  Pendine 
facility. 

•  Benefit:  The  Center’s  advice  helped  Defence  Science  and 
Technology  Laboratory  to  improve  the  quality  of  TSPI 
collected  during  the  event  while  minimizing  the  cost  to  the 
program. 
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Navy:  Distributed  Aperture  Infrared  Countermeasure  2 

(DAIRCM2)  Laser  Warning,  Phase  1 

•  Sponsors:  Naval  Research  Laboratory 

•  Activity:  The  Center  conducted  static  ground  tests  of  the  laser 
warning  function  in  the  DAIRCM2  sensors. 

•  Benefit:  The  data  collected  from  this  effort  allowed  the 
sponsor  to  improve  laser  detection  algorithms  and  reduce  the 
risks  associated  with  a  follow-on  flight  test  of  the  DAIRCM 
system. 

ROTARY-WING  AND  FIXED-WING  TEST  EVENTS 

Air  Force:  Large  Aircraft  Infrared  Countermeasures 

(LAIRCM)  Next  Generation  HC/MC-130J  Flight  Test 

•  Sponsor:  46th  Test  Wing  Test  Squadron,  Defensive  Systems 
and  Mobility  Directorate,  Air  Force  Life  Cycle  Management 
Center 

•  Activity:  The  Center  provided  one  Joint  Mobile  IRCM  Testing 
System  (JMITS),  one  Multi-spectral  Sea  and  Land  Target 
Simulator  (MSALTS)  missile  simulator,  and  personnel  to 
perform  two-color,  infrared  (1R)  simulations  to  collect  system 
response  data  to  assess  the  LAIRCM  system,  as  installed  on 
the  HC/MC-130J.  The  Air  Force  conducted  the  test  at  Eglin 
Air  Force  Base  (AFB),  Florida. 

•  Benefit:  The  testing  provided  the  Air  Force  with  a 
cost-effective  test  venue  to  collect  critical  data  needed  to 
assess  performance  of  the  LAIRCM  system  installed  on  a  new 
platform,  the  HC/MC-130J. 

Air  Force:  LAIRCM  System  Processor  Replacement  Altitude 

Reference  Unit  Replacement  Flight  Test 

•  Sponsor:  46th  Test  Wing  Test  Squadron,  Defensive  Systems 
and  Mobility  Directorate,  Air  Force  Life  Cycle  Management 
Center 

•  Activity:  The  Center  provided  an  MSALTS  missile  simulator 
and  personnel  to  perform  two-color,  IR,  and  ultra-violet  (UV) 
simulations  to  collect  system  response  data  needed  to  assess 
the  upgraded  system  software  with  the  new  altitude  reference 
unit.  The  Air  Force  conducted  the  tests  at  Eglin  AFB,  Florida. 

•  Benefit:  The  testing  provided  the  Air  Force  with  critical  data 
needed  to  assess  performance  of  the  upgraded  LAIRCM 
system. 

Army:  Reduced  Optical  Signature  Emissions  Solution 

Infrared  CM  Test  8.1 

•  Sponsors:  U.S.  Army  Technology  Application  Program  Office 
(TAPO)  and  160th  Special  Operations  Aviation  Regiment 
Systems  Integration  and  Maintenance  Office 

•  Activity:  The  Center  provided  subject  matter  expertise  and 

a  JMITS  van  with  four  reactive-configured  1R  seekers  during 
the  IRCM  effectiveness  test  for  the  MH-60M  and  MH-47G 
aircraft.  These  tests  evaluated  new  flare  CM  sequences  and 
variations  of  current  flare  CM  sequences  using  improved 
flares,  or  different  flares  within  the  sequences.  The  Center 
provided  near  real-time  data  reduction  and  flare  sequence 
analysis  to  assist  the  sponsor  in  making  test  decisions  on  flare 
sequence  performance  and  to  provide  recommendations  on 
flare  sequence  timing  and/or  pattern  adjustments.  After  the 


test,  the  Center  provided  an  independent  assessment  and  a 
briefing  of  test  results  to  TAPO  leadership. 

•  Benefit:  The  data  collected  from  this  effort  allowed  TAPO  to 
use  the  test  results  to  procure  new  flares  needed  to  enhance 
protection  of  the  MH-60M  and  MH-47  aircraft  against  IR  Man 
Portable  Air  Defense  Systems  (MANPADS). 

Army:  Evaluation  of  IRCM  during  Missile  Simulations 

Flight  Testing 

•  Sponsor:  U.S.  Army  Project  Management  Office,  Aircraft 
Survivability  Equipment  (PMO-ASE),  Program  Manager, 
Countermeasures  Sensors 

•  Activity:  The  Center  provided  a  JMITS  for  UV  simulations, 
four  reactive-configured  IR  seekers,  and  subject  matter 
expertise  during  the  testing  for  the  UH-60M,  AH-64E,  and 
CH-47F  aircraft.  These  tests  evaluated  the  performance  of 
each  IRCM  (laser  jammer  and/or  flare  sequence)  installed  on 
the  aircraft  against  the  static  IR  seekers. 

•  Benefit:  The  data  collected  from  this  effort  allowed 
PMO-ASE  to  assess  the  performance  of  the  common  missile 
warning  sensor  and  IRCM  installed  on  the  UH-60M,  AH-64E, 
and  CH-47F  aircraft. 

Navy:  MV-22  Universal  Urgent  Needs  Statement, 

Department  of  the  Navy  (DoN),  LAIRCM  Integration  Test 

Phase  1 

•  Sponsor:  Navy  Program  Executive  Office  (PEO),  Advanced 
Tactical  Aircraft  Protection  Systems  Program  Office 

•  Activity:  The  Center  provided  the  MSALTS  two-color 
IR  missile  simulations,  along  with  jam  beam  radiometers, 
threat-representative  laser  beamriders,  a  designator, 
rangefinder,  and  radar  threat  simulations  using  the  PRTS. 

•  Benefit:  The  testing  provided  the  critical  data  needed  to 
support  a  fleet  introduction  decision  for  the  DoN  LAIRCM 
Advanced  Threat  Warning  (ATW)  as  installed  on  the  U.S. 
Navy  MV-22  aircraft. 

Navy:  Verification  of  Correction  of  Deficiencies  Test  of  the 

CH-53E  DoN  LAIRCM  ATW  Sensor 

•  Sponsor:  Navy  PEO,  Advanced  Tactical  Aircraft  Protection 
Systems  Program  Office 

•  Activity:  The  Center  provided  the  MSALTS  two-color  IR 
missile  simulators  and  jam  beam  radiometers. 

•  Benefit:  The  testing  allowed  the  Navy  to  collect  critical  data 
needed  to  assess  the  performance  of  the  DoN  LAIRCM  ATW 
hardware  and  software  upgrades. 

Navy:  Follow-On  Operational  T&E  of  the  CH-53E  DoN 

LAIRCM  ATW  Sensor  Phase  I  &  II 

•  Sponsor:  Navy  PEO,  Advanced  Tactical  Aircraft  Protection 
Systems  Program  Office 

•  Activity:  The  Center  provided  the  MSALTS  two-color 
IR  missile  simulations,  along  with  jam  beam  radiometers, 
threat-representative  laser  beamriders,  a  designator,  and 
rangefinder  systems  during  the  CH-53E  DoN  LAIRCM  ATW 
system  flight  testing. 

•  Benefit:  The  testing  provided  a  cost-effective  test  venue  for 
collecting  critical  missile  warning  sensor  and  laser  warning 
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sensor  data  needed  to  evaluate  and  assess  the  readiness  of  the 
CH-53E  DoN  LA1RCM  ATW  system  for  fleet  deployment  in 
theatre. 

Navy:  P-8A  Poseidon  LAIRCM  Flight  Test 

•  Sponsor:  Navy  Air  T&E  Squadron  20 

•  Activity:  The  Center  provided  the  MSALTS  two-color 
1R  missile  simulator  for  flight  testing  of  the  P-8  A  ASE. 

The  Center  provided  all  data  collected  to  the  sponsors  for 
assessment. 

•  Benefit:  The  testing  provided  the  Navy  with  the  data 
necessary  to  assess  the  performance  of  the  LAIRCM  system  as 
installed  on  the  P-8A. 

Navy:  KC-130J  Integration  Flight  Test 

•  Sponsor:  Navy  PEO,  Advanced  Tactical  Aircraft  Protection 
Systems  Program  Office 

•  Activity:  The  Center  provided  MSALTS  two-color  IR  missile 
simulations  during  flight  testing  of  the  KC-130J  Integration 
Flight  Test.  The  Center  provided  all  data  collected  to  the 
sponsor  for  assessments. 

•  Benefit:  The  testing  provided  the  critical  data  needed  to 
support  a  fleet  introduction  decision  for  the  DoN  LAIRCM 
ATW  as  installed  on  the  KC-130J  aircraft. 


National  Atlantic  Treaty  Organization  (NATO):  Trial  MACE 

XVI 

•  Sponsor:  The  Center/JCMT&E  WG 

•  Activity:  The  Center  provided  three  analysts  to  help  process 
data  and  produce  reporting  products  during  Trial  MACE  XVI 
at  the  Military  Training  Area  in  Lest’,  Slovakia. 

•  Benefit:  Trial  MACE  provided  the  Center  and  DOD  with  an 
opportunity  to  understand  the  current  NATO  radio-frequency 
(RF)  test  methodologies  and  to  review  actual  threat 
capabilities. 

LIVE  FIRE  TEST  EVENTS 

Navy:  DoN  LAIRCM  Dugway  Live  Fire  2015 

•  Sponsor:  Navy  PEO,  Advanced  Tactical  Aircraft  Protection 
Systems  Program  Office 

•  Activity:  The  Center  provided  MSALTS  UV  and  two-color 
IR  missile  simulations  and  laser  threats  to  support  updates  to 
the  DoN  LAIRCM  system,  MV-22,  KC-130J,  and  CH-53E 
configurations. 

•  Benefit:  The  testing  provided  critical  data  needed  to  assess 
DON  LAIRCM  missile  and  laser  warning  performance  against 
various  threats,  including  missiles  in  free  flight,  and  to  support 
the  Validation  and  Verification  of  the  Digital  System  Module. 


PGW  CM  ACTIVITIES 


Army:  Joint  Air-to-Ground  Missile  (JAGM)  Obscurants 

Test 

•  Sponsor:  U.S.  Army  JAGM  Product  Office 

•  Activity:  The  Center,  in  conjunction  with  the  sponsor  and 
the  Army  Missile  and  Aviation  Research  and  Development 
Center,  coordinated,  directed,  and  conducted  tower-mounted 
seeker  tests  of  the  JAGM  seeker  in  obscurant  environments 
against  static  ground  targets. 

•  Benefit:  This  effort  was  designed  to  mature  seeker  tactical 
designs  supporting  the  Tec  lino  logy  Development  program 
and  provided  an  opportunity  for  the  JAGM  Product  Office 
to  verify  the  modeling  and  simulation  tools  for  JAGM. 


OSD:  Vigilant  Hammer  1 

•  Sponsor:  Assistant  Secretary  of  Defense  for  Research  & 
Engineering 

•  Activity:  The  Center  participated  in  a  Joint  Electronic 
Advanced  Technology  RF  experiment.  The  Center  provided  a 
Millimeter-wave  Electronic  Attack  Simulator  to  support  range 
detection  and  susceptibility  experiments  with  the  Naval  Air 
Warfare  Center,  Weapons  Division. 

•  Benefit:  Including  the  Millimeter-wave  Electronic  Attack 
Simulator  in  the  experiment  helped  create  a  complex  and 
dense  RF  environment  that  challenges  participant  systems  to 
detect,  classify,  and  geo-locate  emitters. 


CM-BASED  PRE-DEPLOYMENT  TRAINING  FOR  SERVICE  MEMBER  EXERCISES 


1-6  CAVALRY  MANPADS  RF  Training  -  Fort  Riley,  Kansas 
Joint  Forcible  Entry/Advanced  Integration  -  Nellis  AFB, 
Nevada 

3D  Marine  Aircraft  Wing  (Part  1)  -  Camp  Pendleton, 

California 

Red  Flag  15-1  -  Nellis  AFB,  Nevada 
Red  Flag  15-2  -  Nellis  AFB,  Nevada 

509th  Weapons  Squadron  KC-135  Support  (Part  2)  -  Roswell, 
New  Mexico 

Emerald  Warrior  15  -  Hurlburt  Field,  Florida 
3D  Marine  Aircraft  Wing  (Part  2)  -  Camp  Pendleton, 

California 

•  Sponsors:  Various 

•  Purpose:  The  Center’s  equipment  and  personnel  provided  a 
simulated  threat  environment  and  subject  matter  expertise  to 


observe  aircraft  sensor/ASE  systems  and  crew  reactions  to  this 
environment.  Specifically,  the  Center  emphasized  simulated 
MANPADS  and  RF  threat  engagements  for  participating 
aircraft.  The  Center  also  provided  MANPADS  capabilities 
and  limitations  briefings  to  pilots  and  crews  and  conducted 
“hands-on”  training  at  the  end  of  the  briefings. 

•  Benefit:  These  exercises  provided  realism  to  the  training 
threat  environment  for  the  Service  member  pilots  and  crews  to 
facilitate  understanding  and  use  of  CM  equipment,  especially 
ASE.  The  Center  provided  the  data  collected  to  the  trainers  to 
assist  units  in  developing  and  refining  techniques,  tactics,  and 
procedures  to  enhance  survivability. 


CCM  421 


F  Y 1 5  CENTER  FOR  COUNTERMEASURES 


T&E  TOOLS 


The  Center  has  continued  to  develop  tools  for  1RCM  systems 
T&E  funded  by  the  Undersecretary  of  Defense  (Acquisition, 
Technology  and  Logistics);  the  Test  Resource  Management 
Center;  and  the  Central  T&E  Investment  Program.  Currently,  the 
Center  is  leading  the  development  of  the  MSALTS  and  the  Joint 
Standard  Instrumentation  Suite  (JSIS). 

•  The  MSALTS  is  a  small,  mobile  missile  simulator  that  can  fire 
while  moving  and  simulate  all  current.  Tier  1  missile  threats. 
The  Center  has  designed  the  MSALTS  to  provide  simulated 
signatures  for  the  new  and  more  capable  missile-warning 
systems,  such  as  LA1RCM  Next  Generation,  DoN  LA1RCM, 
and  the  DoN  ATW.  The  Center  initiated  development  of  the 
first  two  systems  in  FY11  and  the  third  system  in  FY12.  The 
developer  completed  fabrication,  assembly,  integration,  and 
government  acceptance  testing  in  December  2014.  The  Center 
successfully  transitioned  all  three  systems  in  1QFY15  to  reach 
an  Initial  Operational  Capability.  In  October  2014,  the  Center 
proposed  an  enhancement  of  the  UV  emitter  to  Undersecretary 
of  Defense  (Acquisition,  Technology  and  Logistics)  to 
support  Common  IRCM  System  operational  testing  slated  for 
2QFY 17.  The  Common  IRCM  System  is  a  component  of  the 
integrated  IRCM  suite  planned  to  defend  U.S.  aircraft  from 
advanced  IR-guided  missiles.  The  system  will  initially  be 
employed  on  the  Army  UH-60  and  Marine  MV-22  aircrafts. 

•  The  JSIS  is  a  transportable,  fully-integrated  instrumentation 
suite  that  will  be  used  to  collect  signature,  TSPI,  acoustic, 
and  related  threat  missile  and  hostile  fire  munitions  metadata. 
The  transportability  of  JSIS  will  allow  it  to  be  used  both  in 
the  United  States  and  abroad  to  reduce  costs  and  expand  the 
types  of  threat  data  available  in  the  United  States.  The  JSIS 
has  been  endorsed  by  the  U.S.  Navy  (PMA  -  272),  Army 
(PMO  -  ASE),  and  the  Air  Force  (LA1RCM  System  Program 
Office)  and  will  be  an  integral  part  of  each  Program  Office’s 
ASE  development.  In  FY14,  the  Center  partnered  with 

the  Arnold  Engineering  Development  Center  and  actively 
created  program  plans,  refined  requirements  from  the  ASE 
T&E  community,  created  and  refined  a  concept  of  operations, 
and  began  identifying  specific  instrumentation  that  meets 
JSIS  requirements.  The  Center  conducted  a  successful 
Critical  Design  Review  in  May  2014.  In  FY15,  the  Center 
conducted  two  integrated  project  reviews  to  check  the  status 
of  the  technical  performance,  schedule,  and  financial  health 


of  the  development.  Development  of  an  Initial  Operational 
Capability  is  expected  to  be  completed  in  FY16,  with  a  risk 
reduction  demonstration  slated  for  1QFY16.  In  FY15,  the 
Center  developed  and  received  the  Doppler  Scoring  Radar 
under  the  JSIS  program.  The  radar  is  capable  of  providing 
three-dimensional  TSPI  on  hostile,  live-fire  activities, 
including  small  arms,  anti-aircraft,  rockets,  and  MANPADS. 
The  system  also  could  be  used  to  report  position  information 
on  aircraft  flight  tracks. 

Additionally,  as  a  result  of  an  internal  electronic  warfare  study 
conducted  by  the  Center  in  FY13,  and  the  increasing  demands 
for  test  tools  that  support  multi-spectral,  integrated  ASE  threat 
environments,  the  Center  internally  funded  the  procurement 
of  two  RF  threat  emitters.  A  low-powered  PRTS  system  was 
delivered  and  started  validation  testing  in  late  FY15  and  a 
high-powered  PRTS  capability  is  scheduled  to  be  delivered 
in  early  FY16.  These  systems  are  being  designed  to  replicate 
short-range  acquisition  and  targeting  radar  systems.  Both 
systems  will  be  validated  to  support  operational  testing  of  the 
APR-39  B/D. 

The  Center  continues  to  develop  and  improve  tools  for  threat, 
live-fire  IRCM  testing.  In  FY14,  the  Missile  and  Space 
Intelligence  Center  began  development  on  a  new  remote-missile 
launcher  for  the  Center.  This  launcher  system  was  developed  to 
support  remote  firing  of  larger  vehicle-launched  IR  surface-to-air 
missiles.  The  system  was  delivered  and  operationally  deployed 
in  FY15  for  a  number  of  live -fire  events.  In  FY16,  continued 
improvements  will  be  considered  to  increase  the  number  of  threat 
types  due  to  sponsor’s  requests. 

The  Center  continued  leading  the  development  of  the  Hostile 
Fire  Signature  (HSIG)  model  enhancements  to  support  HF1  T&E 
activities.  The  baseline  HSIG  Model  project  has  developed 
a  validated,  physics-based  electro-optic  model  that  produces 
signatures  for  the  12.7  mm  Armor  Piercing  Incendiary  Tracer 
round  and  a  rocket-propelled  grenade  (RPG  7)  tracer  and  hard 
body,  sponsored  by  the  Threat  Systems  WG  with  oversight  by 
the  T&E  Threat  Resource  Activity.  The  Center  initiated  spiral 
enhancements  in  1QFY15  to  incorporate  RPG  back  blast  and 
small-arm  muzzle  flash  features  to  the  models.  This  effort  will 
include  data  certification  by  the  Intelligence  Community  and  an 
updated  validation  report. 


JOINT  COUNTERMEASURES  TEST  AND  EVALUATION  WORKING  GROUP 


The  JCMT&E  WG  is  co-chartered  by  DOT&E  and 
DASD(DT&E)  to  measure,  test,  and  assess: 

•  Aircraft  self-protection,  countermeasures,  and  supporting 
tactics 

•  Live-fire  threat  weapons  and  open-air  T&E 

•  System  performance  in  operationally  relevant  aircraft 
installations  and  combat  environments 

•  T&E  methodologies,  instrumentation,  analysis,  and  reporting 


•  Overseas  threat  and  air  electronic  warfare  systems 
performance  and  effectiveness  data  in  coalition  warfare 
environments 

The  JCMT&E  WG  includes  participation  by  DOT&E, 
DASD(DT&E),  all  four  of  the  U.S.  Services,  Australia, 
Canada,  New  Zealand,  the  UK,  and  the  NATO  Air  Force 
Armaments  Group  Sub-Group  2  (SG/2).  The  WG  is  tasked 
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with  actively-seeking  mutually-beneficial  T&E  opportunities  to 
measure  performance  and  suitability  data  necessary  to  provide 
relevant  operational  information  to  deploying  joint/coalition 
Service  members  and  for  U.S.  acquisition  decision  makers. 
Specific  efforts  include: 

•  The  JCMT&E  WG  was  the  U.S.  Technical  Advisor  to  the 
official  negotiations  of  the  Multinational  T&E  Program 
Memorandum  of  Understanding  with  Australia,  Canada, 

New  Zealand,  the  UK,  and  the  United  States  that  was  signed 
into  effect  for  the  U.S.  by  the  HON  Dr.  J.  Michael  Gilmore, 
DOT&E,  in  2015. 

•  The  JCMT&E  WG  conducted  exploratory  meetings  to  identify 
interest  in  developing  bilateral  or  multinational  T&E  program 
agreements  with  the  following  nations  in  order  to  conduct 
mutually  advantageous  development  of  T&E  instrumentation, 
methodology,  and  installed  performance  testing  and  to  measure 
live  threat  weapon  firing  data:  Denmark,  Finland,  Norway, 
Sweden,  Germany,  Italy,  Spain,  and  Switzerland. 

•  The  JCMT&E  WG  worked  with  the  Office  of  the  Deputy 
Assistant  Secretary  of  the  Army  Defense  Exports  and 
Cooperation  to  develop  the  four-nation  Aircraft  Electronic 
Warfare  Cooperative  T&E  Project  Arrangement  to  coalesce 
much  of  the  redundant  testing  conducted  by  Australia,  Canada, 
the  UK,  and  the  U.S.  to  significantly  expand  performance 

and  to  collect  suitability  data  to  improve  aircraft  survivability 


and  reduce  cost  for  all  four  nations.  Initiated  identification  of 
required  T&E  infrastructure,  personal,  training,  and  funding 
required  to  conduct  expected  Project  Arrangement  activities. 

•  The  JCMT&E  WG  worked  with  Australia  to  plan  a  combined 
MANPAD/RF  threat  trial  at  the  Woomera  Test  Range, 

South  Australia,  in  September  2016,  to  further  expand  the 
integrated  ASE  test  methodology  to  the  open-air  environment 
using  captive  seekers  and  actual  and  simulated  emitters  for 
fixed-  and  rotary-wing  aircraft  equipped  with  flares  and 
decoys. 

The  JCMT&E  WG  is  cooperating  with  U.S.  allies  to  provide 
opportunities  that  obtain  and  expand  operationally  relevant 
data  useful  for  U.S.  operating  forces,  programs  of  record, 
and  intelligence  organizations  to  reduce  costs  and  field  new 
capabilities  rapidly.  Of  particular  interest  is  obtaining  validated 
data  on  simultaneous  RF/electro-optical/IR  surface-to-air 
missiles,  HFI,  and  anti-tank  guided  missile  firings  by  active 
air-defense  units  and  test  organizations.  The  JCMT&E  WG  is 
building  on  the  successful  NATO  Trial  PULSATILLA  of  May 
2014,  by  coordinating  live  weapons  firings  in  Bulgaria,  Finland, 
and  Slovakia.  These  efforts  will  provide  measured  operational 
performance  of  actual,  modern,  multi-function  radars  and 
integrated  air  defense  systems  that  will  likely  be  used  against 
U.S.  forces. 
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